Ophthalmology Commons^™

Trustworthy Multimodal Ai For Medical Imaging: Enhancing Diagnosis, Reasoning, And Human-Agent Interaction In Extended Reality, Jai Prakash Veerla

Computer Science and Engineering Dissertations

The transition from traditional microscopy to digital pathology has digitized diagnostic data, yet clinical workflows remain constrained by two-dimensional screens and passive, opaque analysis tools that fail to capture the spatial complexity of biological systems. While Foundation Models now promise to reason across histology and genomics, a critical disconnect persists between the richness of this data and the limited cognitive bandwidth of clinicians, who currently lack the immersive interfaces and trustworthy agents necessary to utilize it effectively. This dissertation presents a unified framework for "Embodied Agentic AI," establishing a pipeline that augments physician capabilities through immersive visualization, robust security, and …

Ophthoacr (Ophthalmology Automated Chart Review): An Ai-Powered Tool For Complete Automation Of Ophthalmology Chart Reviews And Cohort Data Analysis, Karen M. Chen, Kevin W. Chen, Vlad Diaconita, Stanley Chang, Leejee H. Suh

School of Medicine Faculty Publications

Purpose: Retrospective chart reviews in ophthalmology are essential for gaining clinical insights, but they remain labor-intensive and prone to error. Despite digitization through electronic health records, extracting and interpreting lengthy, unstructured patient histories remains challenging, particularly in ophthalmology, which relies heavily on both imaging and text-based reports. We introduce OphthoACR, a Health Insurance Portability and Accountability Act-compliant artificial intelligence (AI)-powered tool for automated chart review and cohort analyses in ophthalmology. Methods: OphthoACR was applied to extract 16 variables of increasing task difficulty from the complete chart histories of 91 patients who underwent secondary intraocular lens surgery at the Columbia University …

Non-Invasive Detection Of Choroidal Melanoma Via Tear-Derived Protein Corona On Gold Nanoparticles: A Machine Learning Approach, Hakimeh Rakhshandeh, Ahmad Nasiraei, Hamid Riazi-Esfahani, Babak Masoomian, Fariba Ghassemi, Mojtaba Arjmand, Saeed Heidari Keshel, Fatemeh Atyabi, Rassoul Dinarvand

Wills Eye Hospital Papers

This study investigates the feasibility of using tear sample analysis, based on protein corona formation on gold nanoparticles combined with electrospray ionization mass spectrometry (ESI-MS) and machine learning techniques, as a non-invasive approach for the detection of choroidal melanoma. The aim is to assess whether protein-nanoparticle interactions can support early and reliable identification of this ocular condition. Tear samples were collected using Schirmer strips from six healthy individuals and six patients diagnosed with choroidal melanoma, with subsequent augmentation to 18 samples per group. Gold nanoparticles (AuNPs, ~ 20 nm) were synthesized via citrate reduction and incubated with tear samples to …

Alphamissense Predictions And Clinvar Annotations: A Deep Learning Approach To Uveal Melanoma, David J. Taylor Gonzalez, Mak B. Djulbegovic, Meghan Sharma, Michael Antonietti, Colin K. Kim, Vladimir N. Uversky, Carol L. Karp, Carol L. Shields, Matthew W. Wilson

Wills Eye Hospital Papers

OBJECTIVE: Uveal melanoma (UM) poses significant diagnostic and prognostic challenges due to its variable genetic landscape. We explore the use of a novel deep learning tool to assess the functional impact of genetic mutations in UM.

DESIGN: A cross-sectional bioinformatics exploratory data analysis of genetic mutations from UM cases.

SUBJECTS: Genetic data from patients diagnosed with UM were analyzed, explicitly focusing on missense mutations sourced from the Catalogue of Somatic Mutations in Cancer (COSMIC) database.

METHODS: We identified missense mutations frequently observed in UM using the COSMIC database, assessed their potential pathogenicity using AlphaMissense, and visualized mutations using AlphaFold. Clinical …

Oculomics: Current Concepts And Evidence, Zhuoting Zhu, Yueye Wang, Ziyi Qi, Wenyi Hu, Xiayin Zhang, Siegfried K Wagner, Yujie Wang, An Ran Ran, Joshua Ong, Ethan Waisberg, Mouayad Masalkhi, Alex Suh, Yih Chung Tham, Carol Y Cheung, Xiaohong Yang, Honghua Yu, Zongyuan Ge, Wei Wang, Bin Sheng, Yun Liu, Andrew G Lee, Alastair K Denniston, Peter Van Wijngaarden, Pearse A Keane, Ching-Yu Cheng, Mingguang He, Tien Yin Wong

Faculty, Staff and Student Publications

The eye provides novel insights into general health, as well as pathogenesis and development of systemic diseases. In the past decade, growing evidence has demonstrated that the eye's structure and function mirror multiple systemic health conditions, especially in cardiovascular diseases, neurodegenerative disorders, and kidney impairments. This has given rise to the field of oculomics-the application of ophthalmic biomarkers to understand mechanisms, detect and predict disease. The development of this field has been accelerated by three major advances: 1) the availability and widespread clinical adoption of high-resolution and non-invasive ophthalmic imaging ("hardware"); 2) the availability of large studies to interrogate associations …

A Hybrid Deep Learning-Based Approach For Visual Field Test Forecasting, Ashkan Abbasi, Sowjanya Gowrisankaran, Wei-Chun Lin, Xubo Song, Bhavna Josephine Antony, Gadi Wollstein, Joel Schuman, Hiroshi Ishikawa

Wills Eye Hospital Papers

OBJECTIVE: Longitudinal assessment of visual field (VF) testing is essential in glaucoma management. Conventional VF forecasting methods require numerous prior tests, while deep learning techniques have shown promising results with fewer tests. This study introduces a hybrid deep learning framework to enhance flexibility and accuracy in VF test forecasting.

DESIGN: A retrospective longitudinal study using deep learning-based VF forecasting models.

SUBJECTS AND CONTROLS: A total of 1750 subjects (healthy and glaucoma patients) with 19 437 Humphrey VF (24-2 Swedish Interactive Threshold Algorithm) tests collected from longitudinal glaucoma cohorts at the University of Pittsburgh and New York University.

METHODS: Three deep …

Artificial Intelligence-Based Methodologies For Early Diagnostic Precision And Personalized Therapeutic Strategies In Neuro-Ophthalmic And Neurodegenerative Pathologies, Rahul Kumar, Ethan Waisberg, Joshua Ong, Phani Paladugu, Dylan Amiri, Jeremy Saintyl, Jahnavi Yelamanchi, Robert Nahouraii, Ram Jagadeesan, Alireza Tavakkoli

SKMC Student Presentations and Publications

Advancements in neuroimaging, particularly diffusion magnetic resonance imaging (MRI) techniques and molecular imaging with positron emission tomography (PET), have significantly enhanced the early detection of biomarkers in neurodegenerative and neuro-ophthalmic disorders. These include Alzheimer's disease, Parkinson's disease, multiple sclerosis, neuromyelitis optica, and myelin oligodendrocyte glycoprotein antibody disease. This review highlights the transformative role of advanced diffusion MRI techniques-Neurite Orientation Dispersion and Density Imaging and Diffusion Kurtosis Imaging-in identifying subtle microstructural changes in the brain and visual pathways that precede clinical symptoms. When integrated with artificial intelligence (AI) algorithms, these techniques achieve unprecedented diagnostic precision, facilitating early detection of neurodegeneration and …

Student Perceptions Of A Novel No-Cost Mobile Application For Ophthalmic History And Physical Examination, Soryan Kumar, Anagha Lokhande, Spandana Jarmale, Arnav Kumar, Samantha Rosenthal, Grayson W. Armstrong, Michael Migliori, Jamie Schaefer

Journal of Academic Ophthalmology

Background: Mobile applications have shown promise in enhancing medical trainee performance. In ophthalmology, a comprehensive mobile app can streamline the trainee education process by providing guidance for patient intake. Language barriers pose additional challenges impacting the quality of care for Spanish-speaking patients; literature has documented the adverse impacts of inadequate translation on quality of medical care for both trainees and patients. We aim to develop a free mobile application to guide medical trainees through the ophthalmic patient intake process and assist with Spanish-language translation.

Methods: We developed EyeCheck as a free mobile application for ophthalmology trainee education with …

Artificial Intelligence And Machine Learning In Ocular Oncology, Retinoblastoma (Armor): Experience With A Multiracial Cohort, Vijitha S. Vempuluru, Rajiv Viriyala, Virinchi Ayyagari, Komal Bakal, Patanjali Bhamidipati, Krishna Krishore Dhara, Sandor R. Ferenczy, Carol L. Shields, Swathi Kaliki

Wills Eye Hospital Papers

Background: The color variation in fundus images from differences in melanin concentrations across races can affect the accuracy of artificial intelligence and machine learning (AI/ML) models. Hence, we studied the performance of our AI model (with proven efficacy in an Asian-Indian cohort) in a multiracial cohort for detecting and classifying intraocular RB (iRB). Methods: Retrospective observational study. Results: Of 210 eyes, 153 (73%) belonged to White, 37 (18%) to African American, 9 (4%) to Asian, 6 (3%) to Hispanic races, based on the U.S. Office of Management and Budget's Statistical Policy Directive No.15 and 5 (2%) had no reported race. …

Transformer-Based Deep Learning Prediction Of 10-Degree Humphrey Visual Field Tests From 24-Degree Data, Min Shi, Anagha Lokhande, Yu Tian, Yan Luo, Mohammad Eslami, Saber Kazeminasab, Tobias Elze, Lucy Shen, Louis Pasquale, Sarah Wellik, Carlos Gustavo De Moraes, Jonathan Myers, Nazlee Zebardast, David Friedman, Michael Boland, Mengyu Wang

Wills Eye Hospital Papers

PURPOSE: To predict 10-2 Humphrey visual fields (VFs) from 24-2 VFs and associated non-total deviation features using deep learning.

METHODS: We included 5189 reliable 24-2 and 10-2 VF pairs from 2236 patients, and 28,409 reliable pairs of macular OCT scans and 24-2 VF from 19,527 eyes of 11,560 patients. We developed a transformer-based deep learning model using 52 total deviation values and nine VF test features to predict 68 10-2 total deviation values. The mean absolute error, root mean square error, and the R2 were evaluation metrics. We further evaluated whether the predicted 10-2 VFs can improve the structure-function relationship …

Predicting Choroidal Nevus Transformation To Melanoma Using Machine Learning, Prashant D. Tailor, Piotr K. Kopinski, Haley S. D'Souza, David A. Leske, Timothy W. Olsen, Carol L. Shields, Jerry A. Shields, Lauren A. Dalvin

Wills Eye Hospital Papers

PURPOSE: To develop and validate machine learning (ML) models to predict choroidal nevus transformation to melanoma based on multimodal imaging at initial presentation.

DESIGN: Retrospective multicenter study.

PARTICIPANTS: Patients diagnosed with choroidal nevus on the Ocular Oncology Service at Wills Eye Hospital (2007-2017) or Mayo Clinic Rochester (2015-2023).

METHODS: Multimodal imaging was obtained, including fundus photography, fundus autofluorescence, spectral domain OCT, and B-scan ultrasonography. Machine learning models were created (XGBoost, LGBM, Random Forest, Extra Tree) and optimized for area under receiver operating characteristic curve (AUROC). The Wills Eye Hospital cohort was used for training and testing (80% training-20% testing) with …

A Comparative Study Of Responses To Retina Questions From Either Experts, Expert-Edited Large Language Models, Or Expert-Edited Large Language Models Alone, Prashant D. Tailor, Lauren A. Dalvin, John J. Chen, Raymond Iezzi, Timothy W. Olsen, Brittni A. Scruggs, Andrew J. Barkmeier, Sophie J. Bakri, Edwin H. Ryan, Peter H. Tang, D. Wilkin Parke, Peter Belin, Jayanth Sridhar, David Xu, Ajay E. Kuriyan, Yoshihiro Yonekawa, Matthew R. Starr

Wills Eye Hospital Papers

OBJECTIVE: To assess the quality, empathy, and safety of expert edited large language model (LLM), human expert created, and LLM responses to common retina patient questions.

DESIGN: Randomized, masked multicenter study.

PARTICIPANTS: Twenty-one common retina patient questions were randomly assigned among 13 retina specialists.

METHODS: Each expert created a response (Expert) and then edited a LLM (ChatGPT-4)-generated response to that question (Expert + artificial intelligence [AI]), timing themselves for both tasks. Five LLMs (ChatGPT-3.5, ChatGPT-4, Claude 2, Bing, and Bard) also generated responses to each question. The original question along with anonymized and randomized Expert + AI, Expert, and LLM …

Enabling Customization Of Discussion Forums For Blind Users, Mohan Sunkara, Yash Prakash, Hae-Na Lee, Sampath Jayarathna, Vikas Ashok

Computer Science Faculty Publications

Online discussion forums have become an integral component of news, entertainment, information, and video-streaming websites, where people all over the world actively engage in discussions on a wide range of topics including politics, sports, music, business, health, and world affairs. Yet, little is known about their usability for blind users, who aurally interact with the forum conversations using screen reader assistive technology. In an interview study, blind users stated that they often had an arduous and frustrating interaction experience while consuming conversation threads, mainly due to the highly redundant content and the absence of customization options to selectively view portions …

Autodesc: Facilitating Convenient Perusal Of Web Data Items For Blind Users, Yash Prakash, Mohan Sunkara, Hae-Na Lee, Sampath Jayarathna, Vikas Ashok

Computer Science Faculty Publications

Web data items such as shopping products, classifieds, and job listings are indispensable components of most e-commerce websites. The information on the data items are typically distributed over two or more webpages, e.g., a ‘Query-Results’ page showing the summaries of the items, and ‘Details’ pages containing full information about the items. While this organization of data mitigates information overload and visual cluttering for sighted users, it however increases the interaction overhead and effort for blind users, as back-and-forth navigation between webpages using screen reader assistive technology is tedious and cumbersome. Existing usability-enhancing solutions are unable to provide adequate support in …

Detecting Deceptive Dark-Pattern Web Advertisements For Blind Screen-Reader Users, Satwick Ram Kodandaram, Mohan Sunkara, Sampath Jayarathna, Vikas Ashok

Computer Science Faculty Publications

Advertisements have become commonplace on modern websites. While ads are typically designed for visual consumption, it is unclear how they affect blind users who interact with the ads using a screen reader. Existing research studies on non-visual web interaction predominantly focus on general web browsing; the specific impact of extraneous ad content on blind users' experience remains largely unexplored. To fill this gap, we conducted an interview study with 18 blind participants; we found that blind users are often deceived by ads that contextually blend in with the surrounding web page content. While ad blockers can address this problem via …

Eye Movement And Pupil Measures: A Review, Bhanuka Mahanama, Yasith Jayawardana, Sundararaman Rengarajan, Gavindya Jayawardena, Leanne Chukoskie, Joseph Snider, Sampath Jayarathna

Computer Science Faculty Publications

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first …

Leveraging Big Data For Pattern Recognition Of Socio-Demographic And Climatic Factors In Correlation With Eye Disorders In Telangana State, India, Amna Alalawi, Les Sztandera, Parth Lalakia, Anthony Vipin Das, Sai Prashanthi Gumpili, Richard Derman

Kanbar College Faculty Papers

Purpose: Big data is the new gold, especially in health care. Advances in collecting and processing electronic medical records (EMR) coupled with increasing computer capabilities have resulted in an increased interest in the use of big data in health care. Ophthalmology has been an area of focus where results have shown to be promising. The objective of this study was to determine whether the EMR at a multi-tier ophthalmology network in India can contribute to the management of patient care, through studying how climatic and socio-demographic factors relate to eye disorders and visual impairment in the State of Telangana.

Methods: …

The Present And Future Of Artificial Intelligence In Ophthalmology, Robert Abishek, Elliot Cherkas

inSIGHT

Dr. Ravi Goel is a comprehensive ophthalmologist and cataract surgeon at Wills Eye Hospital, with a specific interest in finding ways that AI can help ophthalmologists improve their clinical care and treat more patients. Dr. Goel also publishes a daily blog, Protecting Sight, where he discusses a variety of topics ranging from advances in cataract surgery to medical education. One common thread throughout his blog is the burgeoning impact of AI on the field of ophthalmology, such as the utility of deep learning algorithms for diagnosing various diseases and the impact that improved intra-ocular lens (IOL) power calculations will have …

Endothelial Cell Capture Of Heparin-Binding Growth Factors Under Flow, Bing Zhao, Changjiang Zhang, Kimberly Forsten-Williams, Jun Zhang, Michael Fannon

Ophthalmology and Visual Science Faculty Publications

Circulation is an important delivery method for both natural and synthetic molecules, but microenvironment interactions, regulated by endothelial cells and critical to the molecule's fate, are difficult to interpret using traditional approaches. In this work, we analyzed and predicted growth factor capture under flow using computer modeling and a three-dimensional experimental approach that includes pertinent circulation characteristics such as pulsatile flow, competing binding interactions, and limited bioavailability. An understanding of the controlling features of this process was desired. The experimental module consisted of a bioreactor with synthetic endothelial-lined hollow fibers under flow. The physical design of the system was incorporated …