Evolving Perception: Advancing Multimodal and Cross-modal Intelligence for Human Sensory Enhancement at AI-LAMP

Abstract

Multimodal and cross-modal artificial intelligence has immense potential to perceive complex environments and augment human sensory experiences by integrating information across vision, sound, and neural signals. This talk will explore ongoing research in the Artificial Intelligence Laboratory for Augmented Multimodal Perception (AI-LAMP) at San Francisco State University, which advances deep learning frameworks for human-centered sensory applications. In this talk Dr. Ghose will demonstrate AI-LAMP’s research projects in developing perception-aligned, cross-modal intelligence that enhance human capability, empathy, and accessibility. These will include AI-driven visual-to-sound synthesis for multimedia applications; adaptive multimodal system development for real-time navigation assistance to visually impaired users; cross-modal learning for emotion recognition; computer vision-based sensor-independent locomotion control to enhance robotic and mobility systems; EEG-based brain-to-speech translation to support individuals with speech impairments. Collectively, these research aim to redefine how machines perceive and interact with the world, offering powerful opportunities to augment human sensory experience.

Speaker Biography

Sanchita Ghose is an Assistant Professor in the Computer Engineering Department at San Francisco State University since Fall 2022. She is the Director of the AI Lab for Augmented Multimodal Perception (AI-LAMP). She received her Ph.D. and Master’s degree in Electrical Engineering from the University of Texas, San Antonio, in 2022. Prior to joining SFSU, she was a Graduate Research and Teaching Assistant at the University of Texas, San Antonio. Her research interests include deep learning for multimodal applications, computer vision, video action recognition, sound synthesis, and sensory augmentation with cross-modal learning. She is a principal investigator and co-investigator on NSF and DoD-funded research projects. Dr. Ghose’s awards and honors include “Women in Cloud” (2023), the ECE Outstanding Graduate Research Award (2022), the Competitive ECE Pioneer Award (2021), and the Best Research Award in AI Summit (Texas, 2019).