Scratch-based Reflection Art via Differentiable Rendering

Pengfei Shen1*, Ruizeng Li1*, Beibei Wang2,3†, Ligang Liu1†

1University of Science and Technology of China

2Nankai University

3Nanjing University of Science and Technology

*Joint first author  †Corresponding author

ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2023)

Paper
Scratch-based Reflection Art via Differentiable Rendering

Abstract

The 3D visual optical arts create fascinating special effects by carefully designing interactions between objects and light sources. One of the essential types is 3D reflection art, which aims to create reflectors that can display different images when viewed from different directions. Existing works produce impressive visual effects. Unfortunately, previous works discretize the reflector surface with regular grids/facets, leading to a large parameter space and a high optimization time cost. In this paper, we introduce a new type of 3D reflection art - scratch-based reflection art, which allows for a more compact parameter space, easier fabrication, and computationally efficient optimization. To design a 3D reflection art with scratches, we formulate it as a multi-view optimization problem and introduce differentiable rendering to enable efficient gradient-based optimizers. For that, we propose an analytical scratch rendering approach, together with a high-performance rendering pipeline, allowing efficient differentiable rendering. As a consequence, we could display multiple images on a single metallic board with only several minutes for optimization. We demonstrate our work by showing virtual objects and manufacturing our designed reflectors with a carving machine.

Video

Summary

We present a new type of 3D visual reflection art - scratch-based reflection art. We derived algorithms that performs differentiable rendering on microstructures and built machine to manufacture real objects.