Running Plenoxel Nerf

Getting Plenoxel to run has been a bit of an adventure. A bit of time was spent getting CUDA and pytorch setup properly. In particular, the environment.yaml would cause conda to install the CPU version of torch. sigh The CUDA version required, 11.3, only supported Ubuntu 20.04. Luckily, I'm using WSL2, so I installed a Ubuntu 20.04, and rebuilt it there. And then I magically (I have no idea how I did it actually), got torch installed with CUDA. 🤷

Another problem is I only have a RTX-2060 mobile GPU with 6GB of RAM. The default width x height is [256,256] and [512,512].

But, as of right now, it's not generating good images. Here's an example.

I don't know how that translates to RAM usage, but obviously 6GB isn't enough. So, I've run it different ways:

python -u opt.py -t lego lego_real_night_radial/ --reso [[32,32,32],[64,64,64]]

This will run and finish, but have artifacts like the image above.

python -u opt.py -t lego lego_real_night_radial/ --reso [[128,128,128],[256,256,256]] -c configs/syn.json
python -u opt.py -t lego lego_real_night_radial/ --reso [[32,32,32],[64,64,64]] -c configs/syn.json
python -u opt.py -t lego lego_real_night_radial/ --reso [[64,64,64],[128,128,128]] -c configs/syn.json
python -u opt.py -t lego lego_real_night_radial/ --reso [[32,32,32],[64,64,64]] -c configs/syn.json
python -u opt.py -t lego lego_real_night_radial/ --reso [[32,32,32],[64,64,64]]

These all run out of memory, still.

There is a command for PyTorch, max_split_size_mb that can try to avoid the OOM error. It can be set with the environment variable, PYTORCH_CUDA_ALLOC_CONF

PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:512 python -u opt.py -t lego lego_real_night_radial/ --reso [[128,128,128],[256,256,256]]

For the default, it still OOMs. But, for smaller cases, it finishes, it just doesn't look right. I don't know enough about this to debug, so I'm going to move on Gaussian Splatting.