Research

My research focuses on AR/VR/MR, computer vision and computational imaging. I study how light, geometry, materials, imaging devices, and learned representations interact with the physical world. I develop projector-camera (ProCam) and spatial augmented reality (SAR) systems that can perceive, simulate, and modify the appearance of real scenes. Related research addresses 3D vision, visual privacy and security, and interdisciplinary visual computing.

This research uses physical image formation to control appearance, recover scene structure, study privacy and security risks, and analyze scientific and biomedical images.

Projector-Camera Systems (ProCams) & Spatial Augmented Reality (SAR)

Research questionHow can projected light become a reliable computational interface between digital content and real-world environments?

We study projector-camera systems as a computational interface for modifying and augmenting the appearance of the physical world. Our work spans projector compensation, relighting, project-and-capture modeling, and Spatial Augmented Reality (SAR). It has progressed from learning-based photometric compensation to unified geometric-photometric modeling, efficient and setup-independent deployment, physically based and 3D-aware ProCam simulation, and intelligent SAR. Across these stages, the central goal is to determine how projected light should be generated and adapted so that digital content can interact reliably with real surfaces, scenes, and users.

Research trajectory

Compensation → Unified ProCam Modeling → Practical Deployment → Physics/3D-Aware ProCams → Generalization → Intelligent SAR

  1. Learning-based compensation

    CompenNet (CVPR'19) CompenNet++ (ICCV'19)

  2. Unified ProCam modeling

    DeProCams (TVCG'21)

  3. Efficient / practical compensation

    Efficiency comparisons and an online video compensation pipeline.
    CompenNeSt++ (TPAMI'22) ViComp (TVCG'24) CompenHR (IEEE VR'23) CompenRT (ICME'25)

  4. Physics- and 3D-aware ProCams

    Differentiable projector-camera representations, physical rendering, and scene simulation.
    DPCS (TVCG'25) GS-ProCams (TVCG'25)

  5. Generalization / modern generative modeling

    Setup-independent projector compensation with pretrained weights and geometric alignment.
    SIComp (TVCG'26) DiffPC (TVCG'26)

  6. Intelligent SAR

    LAPIG (TVCG'25) ProCap (IEEE VR'26)
Publications (12)

Physics-Informed 3D Vision & Computational Imaging

Research questionHow can physical models, geometry, and learned priors help recover scene structure, appearance, and 3D representations from visual measurements?

We study inverse problems in computer vision, recovering scene structure and physical properties from incomplete or distorted measurements. Our work combines active sensing, geometric constraints, optical image formation, differentiable rendering, and learned priors to estimate depth, shape, appearance, and scene representations. Applications include projector-camera calibration, structured-light reconstruction, imaging through challenging optical media, and neural or explicit 3D scene representations. We use physical models to constrain and guide these inference problems.

Research framework

  1. Active Sensing & Measurement

    Adaptive structured illumination, projector-camera calibration, and recovered surface geometry.
    ProCamCalib (TASE'21) Adaptive SL (ISMAR'23)

  2. Physics-Based Image Formation

    Physics-based underwater image formation and camera rays refracted through housing and water interfaces.
    NeuroPump (ACM MM'25) DPCS (TVCG'25) Neural-STE (AAAI'21)

  3. Physics-Guided Reconstruction

    DeProCams (TVCG'21) NeuroPump (ACM MM'25)

  4. 3D Representation & Synthesis

    Multiscale spherical grids, foveated rendering, and omnidirectional scene synthesis.
    MSTD (VRW'25) GS-ProCams (TVCG'25)
Publications (8)

Visual Privacy & Physical-World Security

Research questionWhat new attacks and defenses arise when computer vision interacts with physical light transport rather than only digital pixels?

We study privacy and security problems that emerge when machine perception interacts with the physical world. One line investigates information leakage through ordinary optical media and privacy risks introduced by learning-based computational imaging. Another studies projector-based adversarial attacks, where controllable light modifies the appearance observed by vision systems without permanently altering objects. This work progresses from stealthy projector attacks to classifier-agnostic and view-invariant attacks, combining differentiable project-and-capture models with increasingly realistic physical scene representations.

Research trajectory

  1. Physical mail privacy: attack & defense

    Physical mail content recovery and privacy protection examples.
    Neural-STE (AAAI'21)

  2. Stealthy differentiable projector attack

    Stealthy projector-based adversarial attack on image classifiers.
    SPAA (IEEE VR'22)

  3. Classifier-agnostic projector attack

    Classifier-agnostic physical projector attack examples.
    CAPAA (ICME'25)

  4. View-invariant & classifier-agnostic physical attack

    Physical projected attacks observed from multiple viewpoints.
    VIPA (TVCG'26)
Publications (4)

Interdisciplinary collaborations

Scientific & Biomedical Visual Computing

Research questionHow can computer vision extract reliable, interpretable, and useful measurements from specialized scientific and biomedical imagery?

We collaborate with domain experts on scientific and biomedical problems whose data, acquisition conditions, and interpretation requirements differ from conventional vision benchmarks. Our work includes cryo-EM analysis, multi-view biological phenotyping, noninvasive visual classification, and mixed-reality medical training. A recurring goal is to derive reliable structural measurements and interpretable results from these observations through representation learning, multi-view analysis, keypoint-based modeling, and human-centered visualization.

Research topics

  1. Scientific Imaging

    Orientation / Structure

    CryoPoseNet (Acta Crystallographica Section D - Structural Biology'22)

  2. Biological Phenotyping

    Multi-view sturgeon images, attention maps, and morphological landmarks.

    Morphology / Biological Traits

    Phenome (Water Biology and Security'25) Sturgeon sex (Aquaculture'26)

  3. Biomedical MR

    The public 2025 mixed-reality acupoint teaching framework: localization, practice, and evaluation.

    Interactive Training

    MR Training (ISMAR-Adjunct'25)
Publications (4)
Back to top ↑