Fermat Non-line-of-sight scanning

A Novel Theory of Fermat Paths

We present a novel theory of Fermat paths of light between a known visible scene and an unknown object not in the line of sight of a transient camera. These light paths either obey specular reflection or are reflected by the object’s boundary, and hence encode the shape of the hidden object. We prove that Fermat paths correspond to discontinuities in the transient measurements. We then derive a novel constraint that relates the spatial derivatives of the path lengths at these discontinuities to the surface normal. Based on this theory, we present an algorithm, called Fermat Flow, to estimate the shape of the non-line-of-sight object.

Ferma Paths

More Info

Fermat paths are the paths that satisfy Fermat’s principle, meaning they have locally stationary pathlengths. Fermat paths are either specular paths or at object boundary. They are purely geometric, and are invariant to BRDF. In this work, we use Fermat paths for shape reconstruction.

Fermat Flow Constraint

More Info

After detecting Fermat paths and their corresponding pathlengths, what can we tell about the points corresponding to these paths?

First, the point must be on a sphere with the collocated virtual source and detector as its center, and half the Fermat pathlength as its radius. Next, we derive the ray constraint which we call the “Fermat flow” constraint. Then, we conduct sphere-ray intersection to pin down the point.

High Resolution NLOS Imaging

More Info

We use a time-domain, full-frame optical coherent tomography system. We use this system to perform confocal scans under both the looking-around-the-corner and looking-through-diffuser settings. We use spatially and temporally incoherent LED illumination, which allows us to combine transient imaging with diagonal probing. In the context of confocal scanning, this means that we can simultaneously collect transients at all points on the visible surface without scanning, as transient measurements taken at one point will not be contaminated with light emanating from a different point. Our implementation has depth resolution of 10um.

Non-Line-of-Sight Imaging with Fermat Flow

Our method allows, for the first time, accurate shape recovery of complex objects, ranging from diffuse to specular, that are hidden around the corner as well as hidden behind a diffuser. Finally, our approach is agnostic to the particular technology used for transient imaging. As such, we demonstrate mm-scale shape recovery from pico-second scale transients using a SPAD and ultrafast laser, as well as micron-scale reconstruction from femto-second scale transients using interferometry. We believe our work is a significant advance over the state-of-the-art in non-line-of-sight imaging.

Coin Reconstructions


Shumian Xin



Aswin C. Sankaranarayanan



Srinivasa G. Narasimhan



Ioannis Gkioulekas