← All projects

Research project

Projector-Based Adversarial Attacks

Study how physically projected patterns affect image classifiers, from stealthy perturbations to robustness across classifiers and viewpoints.

The problem

A projected pattern can temporarily change the appearance observed by a camera without modifying the object itself. An adversarial projection seeks to change a classifier’s prediction while keeping the physical perturbation subtle and transient. Surface materials, illumination, device responses, and viewpoint changes make a successful digital perturbation difficult to realize physically.

The physical setup connects a projector input, a real surface, a camera observation, and a classifier prediction.

From stealthy attacks to classifier and viewpoint robustness

1. 1. Optimize stealthy physical projections

   Bring a learned project-and-capture model into adversarial and appearance optimization.

   SPAA (IEEE VR'22)

2. 2. Reduce classifier dependence

   Aggregate losses from multiple classifiers and focus perturbations using attention-based gradient weighting.

   CAPAA (ICME'25)

3. 3. Optimize across viewpoints

   Simulate multiple views and jointly optimize adversarial projections across classifiers and camera positions.

   VIPA (TVCG'26)

Stealthy physical projection

SPAA uses PCNet to predict what the camera will capture after a pattern is projected onto the surface. PCNet learns the forward project-and-capture process as a 2D geometric and photometric mapping for a fixed view. The attack optimizer differentiates through this prediction to account for the physical projection process.

The optimization balances adversarial confidence with appearance changes using adversarial and stealthiness losses. This connects the physical projection constraints with the objective of producing subtle targeted or untargeted attacks.

SPAA presentation (short).

Robustness across classifiers

CAPAA addresses dependence on an individual classifier. Its optimization aggregates adversarial and stealthiness gradients from multiple classifiers. Attention-based gradient weighting focuses perturbations on regions with high classification activation and supports robustness under camera-pose variation.

CAPAA connects multiple classifiers with an attention-weighted optimization of the physical projection.

Robustness across viewpoints

VIPA adds explicit multi-view simulation of the projector-camera system. It aggregates simulated attack losses from different camera views and jointly optimizes the adversarial pattern across viewpoints and classifiers before physically projecting the result.

VIPA generates adversarial projections for multiple viewpoints attack.

Papers and implementations

SPAA: Stealthy Projector-Based Adversarial Attacks on Deep Image Classifiers
Bingyao Huang, Haibin Ling
IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), 2022
Paper Code Supplementary Video

CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack
Zhan Li^#, Mingyu Zhao^#, Xin Dong, Haibin Ling, Bingyao Huang
IEEE International Conference on Multimedia and Expo (ICME), 2025
Paper Code

VIPA: View-Invariant Projector-Based Adversarial Attack
Jiyu Han, Qingyue Deng, Bingyao Huang
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2026 (In press)
Also in IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2026
Project Code

Related research topics and projects

Differentiable & 3D ProCam Systems develops the forward simulation models that connect a projector pattern to camera observations. Physics-Informed Computational Imaging includes Neural-STE, which studies passive optical information leakage through envelopes. That privacy problem complements this work, but does not belong to the active projector-attack progression.

Research topics: Visual Privacy & Security

• Differentiable & 3D ProCam Systems
• Physics-Informed Computational Imaging