TRI-STAGE OCCLUSION HANDLING NORMAL MAP ESTIMATION ALGORITHM (TSOHNMEA)

SAMSUNG PRISM PROGRAM

---

This repository is the official implementation of the TSOHNMEA. The project report can be viewed using this link. Complete end-to-end pipeline code is not included yet.

Requirements

To install requirements:

pip install -r requirements.txt

Output

As the end-to-end pipeline code is not included, users must manually take the output from each .ipynb section. The order to extracts and input the outputs is

• background_occlusion_handler.ipynb (not included yet, so we suggest to use an image without any background to test the rest of the pipeline)
• Real_ESRGAN_blur_occlusion.ipynb
• PiFUHD_normal_map_estimation.ipynb

Results

• ORDINARY NORMAL MAP ESTIMATION USING PIFU-HD MODEL:

PIFU-HD.mp4

• OUR TRI-STAGE OCCLUSION HANDLING NORMAL MAP ESTIMATION PIPELINE:

Background, Shadow and Blur occlusion of the original input image is handled and the preprocessed input image is sent to the normal map estimation model

Our-pipeline.mp4

TODO

• Upload complete end-to-end pipeline code (not included right now because further improvements are being made).
• Include background occlusion handler code (not included right now because more advanced and better architectures are being explored).
• Handle object occlusion.

References

https://github.com/tensorflow/models/tree/master/research/deeplab
https://github.com/xinntao/Real-ESRGAN
https://github.com/facebookresearch/pifuhd

Contact

For any queries, feel free to contact at vignesh.nitt10@gmail.com.