Learning Object-Centric Neural Scattering Functions for Free-viewpoint Relighting and Scene Composition
Authors: Hong-Xing Yu, Michelle Guo, Alireza Fathi, Yen-Yu Chang, Eric Ryan Chan, Ruohan Gao, Thomas Funkhouser, Jiajun Wu
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on real and synthetic data show that OSFs accurately reconstruct appearances for both opaque and translucent objects, allowing faithful free-viewpoint relighting as well as scene composition. ... We show qualitative (Figure 6 and Figure 4) and quantitative (Table 1 and Table 2) comparisons on both synthetic and real translucent objects. |
| Researcher Affiliation | Collaboration | 1 Stanford University 2 Google Research |
| Pseudocode | No | The paper describes methods in text and mathematical equations, but does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | Project website with video results: https://kovenyu.com/OSF. The provided link is to a project website focusing on video results, not an explicit statement of code release or a direct link to a source code repository for the methodology. |
| Open Datasets | Yes | We use six synthetic objects, two captured real translucent objects and five real opaque objects from the Diligent-MV dataset (Li et al., 2020). ... We use 5 synthetic objects from the Object Folder (Gao et al., 2021) dataset, as well as a Stanford bunny. |
| Dataset Splits | Yes | For each object, there are 20 views forming a circle around the object. For each view, there are 96 calibrated light sources spatially fixed relative to the camera. We use a front view and a back view (view 10 and view 20) as the testset, and the other 18 views as the training set. ... We split our dataset such that all 20 images of a randomly chosen light direction are held out for testing and all other images for training. ... For each synthetic object, we render 1,000 images, each of which is under a random light direction and a random viewpoint. We sample cameras uniformly on an upper hemisphere, and light directions uniformly on solid angles. We use 500 images for training and 500 for testing. |
| Hardware Specification | No | The paper mentions 'two iPhones 12' for image capture and refers generally to 'GPU memory' requirements, but does not specify any particular GPU/CPU models, processors, or memory amounts used for training or inference of the models. |
| Software Dependencies | No | We generate synthetic images in Blender 3.0, using the Cycles path tracer. We use a standard Structure from Motion (SfM) method, COLMAP (Schonberger & Frahm, 2016). The paper only lists 'Blender 3.0' with a version. Other software like 'COLMAP' is mentioned without a specific version, and core deep learning frameworks are not specified with versions. |
| Experiment Setup | No | We follow the learning strategy from Ne RFs (Mildenhall et al., 2020), which models the pixel color of a ray in an analog to radiance instead of the raw radiometric quantity to simplify learning. ... We leave implementation details and ablation studies in our supplementary material. The main text describes general strategies and the loss function but defers specific hyperparameters, training configurations, or system-level settings to supplementary material. |