Young Achievers Symposium: Yimu Pan

“Cross-Modal Learning and Foundational Models in Medical AI: From Techniques to Applications in Placenta Analysis”

About the Talk:

Cross-modal learning and foundational models represent cutting-edge advancements in medical artificial intelligence (AI), enabling the integration of diverse data modalities—such as medical images and clinical text—to address challenges in data scarcity, heterogeneity, and clinical interpretability. Foundational models, pre-trained on large-scale multimodal datasets, provide a versatile framework for harmonizing information across clinical domains, uncovering latent patterns, and enhancing diagnostic precision. This work explores methodologies for cross-modal alignment, including contrastive learning and attention-based fusion, which empower models to bridge imaging and textual data for comprehensive insights. We demonstrate their application in placenta analysis, a critical yet under-researched domain in prenatal care, where multimodal integration improves the prediction of placental complications.

Generative models, such as diffusion models, are further leveraged to augment scarce medical image datasets, mitigate domain shifts, and enhance model robustness in real-world clinical settings. By synthesizing realistic medical images or generating synthetic clinical narratives, these approaches address data limitations while preserving biological relevance. Our study highlights the synergy of cross-modal learning and foundational models in advancing placenta-centric AI tools, emphasizing their potential to transform prenatal diagnostics, risk stratification, and personalized care. We conclude with practical considerations for deploying such systems in clinical workflows, underscoring the importance of interpretability, ethical data practices, and collaboration with domain experts to ensure translational impact.

Yimu Pan, a doctoral candidate in the College of Information Sciences and Technology at Penn State, will deliver “Cross-Modal Learning and Foundational Models in Medical AI: From Techniques to Applications in Placenta Analysis” as part of CSRAI's Young Achievers Symposium. This lecture is free and open to the Penn State community.