Surface Optimization for NLOS Imaging

Surface vs Volumetric Representation of NLOS Object

Non-line-of-sight (NLOS) imaging is the problem of reconstructing properties of scenes occluded from a sensor, using measurements of light that indirectly travels from the occluded scene to the sensor through intermediate diffuse reflections. We introduce an analysis-by-synthesis framework that can reconstruct complex shape and reflectance of an NLOS object. Our framework deviates from prior work on NLOS reconstruction, by directly optimizing for a surface representation of the NLOS object, in place of commonly employed volumetric representations.

Optimized NLOS Surface Reconstruction

At the core of our framework is a new rendering formulation that efficiently computes derivatives of radiometric measurements with respect to NLOS geometry and reflectance, while accurately modeling the underlying light transport physics. By coupling this with stochastic optimization and geometry processing techniques, we are able to reconstruct NLOS surface at a level of detail significantly exceeding what is possible with previous volumetric reconstruction methods.

Physically Accurate

More Info

We can reconstruct NLOS object shape, in the form of a triangular mesh, and complex reflectance, in the form of a microfacet BRDF, while accurately taking into account the underlying light transport physics.

Computationally Efficient

More Info

This formulation enables the use of Monte Carlo rendering to efficiently estimate derivatives of radiometric measurements with respect to shape and reflectance parameters. We combine an optimized differentiable rendering implementation with stochastic optimization in an inverse rendering framework.

Finer Details

More Info

Through experiments on synthetic and measured data, we show that this pipeline can produce NLOS surface reconstructions at a level of detail comparable to what is achieved by albedo-volume methods using two orders of magnitude more measurements, while additionally recovering non-Lambertian reflectance.

Simulation Result

Surface reconstruction examples: (Top) Ground truth. (Middle) Reconstructions using the light cone transform. (Bottom) Reconstructions from our method.

Experiments with Measured Data

NLOS surface reconstruction using SPAD measurements: (Top) Diffuse object from O’Toole (Middle) A diffuse horse statue. (Bottom) Digit relief on a planar object. In our experiment, we cover the digits with white paper, to increase SNR.


Chia-Yin Tsai



Aswin Sankaranarayanan



Ioannis Gkioulekas



Project Site