Rainbow Learner: Lighting Environment Estimation from a Structural-color based AR Marker

This paper proposes a method for estimating lighting environments from an AR marker coupled with the structural color patterns inherent to a compact disc (CD) form-factor. To achieve photometric consistency, these patterns are used as input to a Conditional Generative Adversarial Network (CGAN), which allows us to efficiently and quickly generate estimations of an environment map. We construct a dataset from pairs of images of the structural color pattern and environment map captured in multiple scenes, and the CGAN is then trained with this dataset. Experiments show that we can generate visually accurate reconstructions with this method for certain scenes, and that the environment map can be estimated in real time.