Ming C. Lin
Reconstructing Reality: Creating Digital Twins for the Metaverse
With increasing availability of data in various forms from images, audio, video, 3D models, motion capture, simulation results, to
satellite imagery, representative samples of the various phenomena constituting the world around us bring new opportunities and
research challenges. Such availability of data has led to recent advances in data-driven modeling. However, most of the existing
example-based synthesis methods offer empirical models and data reconstruction that may not provide an insightful understanding of
the underlying process or may be limited to a subset of observations.
In this talk, I present recent advances that integrate classical model-based methods and statistical learning techniques to tackle
challenging problems that have not been previously addressed. These include flow reconstruction for traffic visualization,
learning heterogeneous crowd behaviors from video, simultaneous estimation of deformation and elasticity parameters from images
and video, and example-based multimodal display for VR systems. These approaches offer new insights for learning and understanding
complex collective behaviors, developing better models for complex dynamical systems from captured data, delivering more effective
medical diagnosis and treatment, as well as cyber-manufacturing of customized apparel. I conclude by discussing some possible
future directions and challenges.