These should have a zero gradient w.r.t the dir when using Empty dir encoding. However, the ultimate camera pose gradient can still have a non-zero gradient w.r.t to the dir.
More precisely, I have trained i-NGP on one image. And now I am trying to get the gradient of I w.r.t u → for all pixels in the image. The process is similar to the train_nerf_step function. However, In this case I define the loss function as follow: L = R + G + B
It is clear that lg.gradient is dloss_by_drgb, but how does the part T * rgb - suffix represent drgb_by_dalpha? I understand that the current color component is calculated by alpha * T * rgb, which produces a derivative T * rgb, but as can be seen in https://github.com/NVlabs/instant-ngp/blob/master/src/testbed_nerf.cu#L1495 , the T is ...
Any use, reproduction, disclosure or * distribution of this software and related documentation without an express * license agreement from NVIDIA CORPORATION is strictly prohibited. */ /** @file envmap.cuh * @author Thomas Müller, NVIDIA * @brief An implementation of kernels for evaluating an envmap and splatting gradients into it. */ #pragma ...
I am saying "approximately", because floating point addition isn't commutative and the order of gradient accumulation depends on thread scheduling, so there can be slight numerical differences. These appear to be somewhat more significant in the NeRF setting than in others.
I even froze the rotation by commenting out the gradient parts through the rotations, and only perturb the translation, and still doesn't recover properly. It's weird because when training the nerf I do allow for extrinsics optimization and it actually doesn't vary much, which means that it should recover a single image pose perturbed very well
// This has the effect of projecting the (already projected) ray gradient from the // tangent plane of the sphere onto the image plane (which is correct!). vec3 image_plane_gradient = inverse (mat3 (xform)) * orthogonal_ray_gradient; // Splat the resulting 2D image plane gradient into the distortion params deposit_image_gradient (image_plane ...
