Physical Sciences and Mathematics Commons^™

Motion-Adjustable Neural Implicit Video Representation, Long Mai, Feng Liu

Computer Science Faculty Publications and Presentations

Implicit neural representation (INR) has been successful in representing static images. Contemporary image-based INR, with the use of Fourier-based positional encoding, can be viewed as a mapping from sinusoidal patterns with different frequencies to image content. Inspired by that view, we hypothesize that it is possible to generate temporally varying content with a single image-based INR model by displacing its input sinusoidal patterns over time. By exploiting the relation between the phase information in sinusoidal functions and their displacements, we incorporate into the conventional image-based INR model a phase-varying positional encoding module, and couple it with a phase-shift generation module …

Novel View Synthesis - A Neural Network Approach, Hoang Le

Dissertations and Theses

Novel view synthesis is an important research problem in computer vision and computational photography. It enables a wide range of applications including re-cinematography, video enhancement, virtual reality, etc. These algorithms leverage a pre-acquired set of images taken from a set of viewpoints to synthesize another image at a novel viewpoint as if it was captured by a real camera. To synthesize a high-quality novel view, these algorithms often assume a static scene, or the images were captured synchronously. However, the scenes in practice are often dynamic, and taking a dense set of images of these scenes at the same moment …

Novel View Synthesis In Time And Space, Simon Niklaus

Dissertations and Theses

Novel view synthesis is a classic problem in computer vision. It refers to the generation of previously unseen views of a scene from a set of sparse input images taken from different viewpoints. One example of novel view synthesis is the interpolation of views in between the two images of a stereo camera. Another classic problem in computer vision is video frame interpolation, which is important for video processing. It refers to the generation of video frames in between existing ones and is commonly used to increase the frame rate of a video or to match the frame rate to …

Good Similar Patches For Image Denoising, Si Lu

Computer Science Faculty Publications and Presentations

Patch-based denoising algorithms like BM3D have achieved outstanding performance. An important idea for the success of these methods is to exploit the recurrence of similar patches in an input image to estimate the underlying image structures. However, in these algorithms, the similar patches used for denoising are obtained via Nearest Neighbour Search (NNS) and are sometimes not optimal. First, due to the existence of noise, NNS can select similar patches with similar noise patterns to the reference patch. Second, the unreliable noisy pixels in digital images can bring a bias to the patch searching process and result in a loss …

Good Similar Patches For Image Denoising (Poster), Si Lu

Computer Science Faculty Publications and Presentations

Patch-based denoising algorithms like BM3D have achieved outstanding performance. An important idea for the success of these methods is to exploit the recurrence of similar patches in an input image to estimate the underlying image structures....

Intensity Inhomogeneity Correction Of Sd-Oct Data Using Macular Flatspace, Andrew Lang, Aaron Carass, Bruno M. Jedynak, Sharon D. Solomon, Peter A. Calabresi, Jerry L. Prince

Mathematics and Statistics Faculty Publications and Presentations

Images of the retina acquired using optical coherence tomography (OCT) often suffer from intensity inhomogeneity problems that degrade both the quality of the images and the performance of automated algorithms utilized to measure structural changes. This intensity variation has many causes, including off-axis acquisition, signal attenuation, multi-frame averaging, and vignetting, making it difficult to correct the data in a fundamental way. This paper presents a method for inhomogeneity correction by acting to reduce the variability of intensities within each layer. In particular, the N3 algorithm, which is popular in neuroimage analysis, is adapted to work for OCT data. N3 works …

Designing In-Headset Authoring Tools For Virtual Reality Video, Cuong Nguyen

Dissertations and Theses

Virtual Reality (VR) video is emerging as a new art form. Viewing VR video requires wearing the VR headset to fully experience the immersive surrounding of the content. However, the novel viewing experience of VR video creates new challenges and requirements for conventional video authoring tools, which were designed mainly for working with normal video on a desktop display. Designing effective authoring tools for VR video requires intuitive video interfaces specific to VR.

This dissertation develops new workflows and systems that enable filmmakers to create and improve VR video while fully immersed in a VR headset. We introduce a series …

Improved Scoring Models For Semantic Image Retrieval Using Scene Graphs, Erik Timothy Conser

Dissertations and Theses

Image retrieval via a structured query is explored in Johnson, et al. [7]. The query is structured as a scene graph and a graphical model is generated from the scene graph's object, attribute, and relationship structure. Inference is performed on the graphical model with candidate images and the energy results are used to rank the best matches. In [7], scene graph objects that are not in the set of recognized objects are not represented in the graphical model. This work proposes and tests two approaches for modeling the unrecognized objects in order to leverage the attribute and relationship models to …

Spatial-Semantic Image Search By Visual Feature Synthesis, Mai Long, Hailin Jin, Chen Fang, Feng Liu

Computer Science Faculty Publications and Presentations

The performance of image retrieval has been improved tremendously in recent years through the use of deep feature representations. Most existing methods, however, aim to retrieve images that are visually similar or semantically relevant to the query, irrespective of spatial configuration. In this paper, we develop a spatial-semantic image search technology that enables users to search for images with both semantic and spatial constraints by manipulating concept text-boxes on a 2D query canvas. We train a convolutional neural network to synthesize appropriate visual features that captures the spatial-semantic constraints from the user canvas query. We directly optimize the retrieval performance …

Bayesian Optimization For Refining Object Proposals, With An Application To Pedestrian Detection, Anthony D. Rhodes

Student Research Symposium

We devise an algorithm using a Bayesian optimization framework in conjunction with contextual visual data for the efficient localization of objects in still images. Recent research has demonstrated substantial progress in object localization and related tasks for computer vision. However, many current state-of-the-art object localization procedures still suffer from inaccuracy and inefficiency, in addition to failing to successfully leverage contextual data. We address these issues with the current research.

Our method encompasses an active search procedure that uses contextual data to generate initial bounding-box proposals for a target object. We train a convolutional neural network to approximate an offset distance …

Cox Processes For Visual Object Counting, Yongming Ma

Student Research Symposium

We present a model that utilizes Cox processes and CNN classifiers in order to count the number of instances of an object in an image. Poisson processes are well suited to events that occur randomly in space, like the location of objects in an image, as well as to the task of counting. Mixed Poisson processes also offer increased flexibility, however they do not easily scale with image size: they typically require O(n3) computation time and O(n2) storage, where n is the number of pixels. To mitigate this problem, we employ Kronecker algebra which takes advantage of the direct product …

Far-Red And Near-Infrared Seminaphthofluorophores For Targeted Pancreatic Cancer Imaging, Lei Wang, Connor W. Barth, Martha Sibrian-Vazquez, Jorge O. Escodedo, Mark Lowry, John Muschler, Haiyan Li, Summer L. Gibbs, Robert Strongin

Chemistry Faculty Publications and Presentations

Molecular probes that selectively highlight pancreatic cancer (PC) tissue have the potential to improve pancreatic ductal adenocarcinoma (PDAC) margin assessment through the selective highlighting of individual PC cells. Herein, we report a simple and unique family of systematically modified red and near-infrared fluorescent probes that exhibit a field-effect-derived redshift. Two of thirteen probes distributed to the normal mouse pancreas following systemic administration. One selectively accumulated in genetically modified mouse models of PDAC. The probe exhibited intracellular accumulation and enabled visualization of four levels of the structure, including the whole organ, resected tissue, individual cells, and subcellular organelles. In contrast to …

Identifying Clouds With Convolutional Neural Networks, Jeff Mullins, Sean Richardson, Peter Drake

Portland Institute for Computational Science Publications

The greatest source of uncertainty in model estimates of projected climate change involve clouds and aerosols. Photographic images of clouds in the sky are simple to acquire and archive, but climate scientists need an automated process for identifying clouds in these images. We bring machine learning to bear on this problem. Specifically, we use convolutional neural networks, which to our knowledge have not previously been applied to this task. We trained a network to identify clear sky, thin cloud, thick cloud, and non-sky pixels in photos taken by the Total Sky Imager. The trained network is capable of classifying 91.9% …

Image Stitching: Handling Parallax, Stereopsis, And Video, Fan Zhang

Dissertations and Theses

Panorama stitching increases the field of view in an image by assembling multiple views together. Traditional stitching techniques are proven to be effective only when dealing with parallax-free monocular images. Many challenges that remain unsolved in the stitching research area include how to stitch monocular images with large parallax, how to stitch stereoscopic images to maintain their stereoscopic consistency and original disparity distribution, and how to create panoramic videos with temporally coherent content. To provide more powerful stitching techniques with more universality, we first develop a parallax-tolerant image stitching technique. With the help of it, we then effectively extend the …

Molecular Probes For Pancreatic Cancer Imaging, Lei Wang

Dissertations and Theses

Pancreatic ductal adenocarcinoma (PDAC) has the poorest five-year survival rate of any cancer. Currently, there are no effective diagnostics or chemotherapeutics. Surgical resection is the only curative therapy. However, most patients experience recurrence due largely to challenges in assessing tumor margin status in the operating room. Molecular probes that selectively highlight pancreatic cancer tissue, having the potential to improve PDAC margin assessment intraoperatively, are urgently needed. In this work, a series of red and near-infrared fluorescent probes is reported. Two were found to distribute to normal pancreas following systemic administration. One selectively accumulates in genetically modified mouse models of PDAC, …

Improved Gain Stability Of A Digital Imager Using A Charge Feedback Amplifier, Elliot Eckman Mylott

Dissertations and Theses

Digital imagers including Charge-Coupled Devices (CCD) are essential to most forms of modern photographic technologies. The quality of the data produced by digital imagers have made them an invaluable scientific measurement tool. Despite the numerous advantages of digital imagers, there are still factors that limit their performance. One such factor is the stability of the camera's gain, the ratio that dictates the imager's ability to convert incident photons to a measurable output voltage. Variations in gain can affect the linearity of the device and produce inaccurate measurements.

One of the factors that determines the gain of the camera is the …

Characterization And Modeling Of Nonlinear Dark Current In Digital Imagers, Justin Charles Dunlap

Dissertations and Theses

Dark current is an unwanted source of noise in images produced by digital imagers, the de facto standard of imaging. The two most common types of digital imager architectures, Charged-Coupled Devices (CCDs) and Complementary Metal-Oxide-Semiconductor (CMOS), are both prone to this noise source. To accurately reflect the information from light signals this noise must be removed. This practice is especially vital for scientific purposes such as in astronomical observations.

Presented in this dissertation are characterizations of dark current sources that present complications to the traditional methods of correction. In particular, it is observed that pixels in both CCDs and CMOS …

Advances In Piecewise Smooth Image Reconstruction, Ralf Juengling

Dissertations and Theses

Advances and new insights into algorithms for piecewise smooth image reconstruction are presented. Such algorithms fit a piecewise smooth function to image data without prior knowledge of the number of regions or the location of region boundaries in the best fitting function. This is a difficult model selection problem since the number of parameters of possible solutions varies widely.

The approach followed in this work was proposed by Yvan Leclerc. It uses the Minimum Description Length principle to make the reconstruction problem well-posed: the best fitting function yields the shortest encoding of the image data. In order to derive a …

Interpreting Individual Classifications Of Hierarchical Networks, Will Landecker, Michael David Thomure, Luis M.A. Bettencourt, Melanie Mitchell, Garrett T. Kenyon, Steven P. Brumby

Computer Science Faculty Publications and Presentations

Hierarchical networks are known to achieve high classification accuracy on difficult machine-learning tasks. For many applications, a clear explanation of why the data was classified a certain way is just as important as the classification itself. However, the complexity of hierarchical networks makes them ill-suited for existing explanation methods. We propose a new method, contribution propagation, that gives per-instance explanations of a trained network's classifications. We give theoretical foundations for the proposed method, and evaluate its correctness empirically. Finally, we use the resulting explanations to reveal unexpected behavior of networks that achieve high accuracy on visual object-recognition tasks using well-known …

Dynamic Ccd Pixel Depletion Edge Model And The Effects On Dark Current Production, Justin Charles Dunlap, Morley M. Blouke, Erik Bodegom, Ralf Widenhorn

Physics Faculty Publications and Presentations

The depletion edge in Charge-Coupled Devices (CCD) pixels is dependent upon the amount of signal charge located within the depletion region. A model is presented that describes the movement of the depletion edge with increasing signal charge. This dynamic depletion edge is shown to have an effect on the amount of dark current produced by some pixels. Modeling the dark current behavior of pixels both with and without impurities over an entire imager demonstrates that this moving depletion edge has a significant effect on a subset of the pixels. Dark current collected by these pixels is shown to behave nonlinearly …

Nonlinear Time Dependence Of Dark Current In Charge-Coupled Devices, Justin Charles Dunlap, Erik Bodegom, Ralf Widenhorn

Physics Faculty Publications and Presentations

It is generally assumed that charge-coupled device (CCD) imagers produce a linear response of dark current versus exposure time except near saturation. We found a large number of pixels with nonlinear dark current response to exposure time to be present in two scientific CCD imagers. These pixels are found to exhibit distinguishable behavior with other analogous pixels and therefore can be characterized in groupings. Data from two Kodak CCD sensors are presented for exposure times from a few seconds up to two hours. Linear behavior is traditionally taken for granted when carrying out dark current correction and as a result, …

Characterization And Correction Of Dark Current In Compact Consumer Cameras, Justin Charles Dunlap, Erik Bodegom, Ralf Widenhorn

Physics Faculty Publications and Presentations

A study of dark current in digital imagers within consumer grade digital cameras is presented. Dark current is shown to vary with temperature, exposure time, and ISO setting. Further, dark current is shown to increase in successive images during a series of images. Consumer cameras are often designed to be as compact as possible and therefore the digital imagers within the camera frame are prone to heat generated by nearby elements within the camera body. It is the scope of this work to characterize the dark current in such cameras and to show that the dark current, in part due …

Study Of The Numerical Modeling Of The Temperature Dependence Of The Dark Current In Charge Coupled Devices, Ralf Widenhorn, Ionel Tunaru, Erik Bodegom, Dan A. Iordache

Physics Faculty Publications and Presentations

As it is well known, the classical works of the Dark Current Spectroscopy method allow - using some not too accurate theoretical relations, but huge numbers of dark current values for thousands of pixels - the evaluation of a reduced number of basic impurities parameters. Unlike these works, this paper tries to obtain--by means of some better approximations of the Shockley-Read-Hall (SRH) model--more information about the studied impurities, as well as the study of the compatibility of the used theoretical model SRH relative to the experimental data. In this manner, both the compatibility SRH model with the studied experimental data …

Influence Of Illumination On Dark Current In Charge-Coupled Device Imagers, Ralf Widenhorn, Ines Hartwig, Justin Charles Dunlap, Erik Bodegom

Physics Faculty Publications and Presentations

Thermal excitation of electrons is a major source of noise in charge-coupled-device (CCD) imagers. Those electrons are generated even in the absence of light, hence, the name dark current. Dark current is particularly important for long exposure times and elevated temperatures. The standard procedure to correct for dark current is to take several pictures under the same condition as the real image, except with the shutter closed. The resulting dark frame is later subtracted from the exposed image. We address the question of whether the dark current produced in an image taken with a closed shutter is identical to the …

Dark Current Behavior In Dslr Cameras, Justin Charles Dunlap, Oleg Sostin, Ralf Widenhorn, Erik Bodegom

Physics Faculty Publications and Presentations

Digital single-lens reflex (DSLR) cameras are examined and their dark current behavior is presented. We examine the influence of varying temperature, exposure time, and gain setting on dark current. Dark current behavior unique to sensors within such cameras is observed. In particular, heat is trapped within the camera body resulting in higher internal temperatures and an increase in dark current after successive images. We look at the possibility of correcting for the dark current, based on previous work done for scientific grade imagers, where hot pixels are used as indicators for the entire chip?s dark current behavior. Standard methods of …

Dark Current Measurements In A Cmos Imager, William C. Porter, Bradley Kopp, Justin Charles Dunlap, Ralf Widenhorn, Erik Bodegom

Physics Faculty Publications and Presentations

We present data for the dark current of a commercially available CMOS image sensor for different gain settings and bias offsets over the temperature range of 295 to 340 K and exposure times of 0 to 500 ms. The analysis of hot pixels shows two different sources of dark current. One source results in hot pixels with high but constant count for exposure times smaller than the frame time. Other hot pixels exhibit a linear increase with exposure time. We discuss how these hot pixels can be used to calculate the dark current for all pixels. Finally, we show that …

Measurements Of Dark Current In A Ccd Imager During Light Exposures, Ralf Widenhorn, Ines Hartwig, Justin Charles Dunlap, Erik Bodegom

Physics Faculty Publications and Presentations

Thermal excitation of electrons is a major source of noise in Charge-Coupled Device (CCD) imagers. Those electrons are generated even in the absence of light, hence the name dark current. Dark current is particularly important for long exposure times and elevated temperatures. The standard procedure to correct for dark current is to take several pictures under the same condition as the real image, except with the shutter closed. The resulting dark frame is later subtracted from the exposed image. We address the question of whether the dark current produced in an image taken with a closed shutter is identical to …

Computation Of Dark Frames In Digital Imagers, Ralf Widenhorn, Armin Rest, Morley M. Blouke, Richard L. Berry, Erik Bodegom

Physics Faculty Publications and Presentations

Dark current is caused by electrons that are thermally exited into the conduction band. These electrons are collected by the well of the CCD and add a false signal to the chip. We will present an algorithm that automatically corrects for dark current. It uses a calibration protocol to characterize the image sensor for different temperatures. For a given exposure time, the dark current of every pixel is characteristic of a specific temperature. The dark current of every pixel can therefore be used as an indicator of the temperature. Hot pixels have the highest signal-to-noise ratio and are the best …

Infrared Response Of Charge-Coupled Devices, Matthias Loch, Ralf Widenhorn, Erik Bodegom

Physics Faculty Publications and Presentations

With a band gap of silicon of 1.1eV, the largest wavelength that can excite electrons from the valence to the conduction band is roughly 1100nm. As a consequence, in, for instance, a charge-coupled device, the quantum efficiency (QE) for wavelengths larger than 1100nm is assumed to be zero. We found that there is a response at those longer wavelengths and that the response decreases with increasing wavelength. The QE increases with increasing chip temperature which suggests a thermally activated process. Impurities in the silicon provide the energy levels in the band gap, from which electrons can be excited either thermally …

Psf Measurements On Back-Illuminated Ccds, Ralf Widenhorn, Alexander Weber, Morley M. Blouke, Albert J. Bae, Erik Bodegom

Physics Faculty Publications and Presentations

The spatial resolution of an optical device is generally characterized by either the Point Spread Function (PSF) or the Modulation Transfer Function (MTF). To directly obtain the PSF one needs to measure the response of an optical system to a point light source. We present data that show the response of a back-illuminated CCD to light emitted from a sub-micron diameter glass fiber tip. The potential well in back-illuminated CCD"s does not reach all the way to the back surface. Hence, light that is absorbed in the field-free region generates electrons that can diffuse into other pixels. We analyzed the …