Multiview Datasets

This repository contains the code for handling several multiview datasets. In particular, it contains the following datasets:

• MP3D-VO Dataset: A dataset of sequence of 360-images for visual odometry task.
• MP3D-FPE Dataset: A dataset for floor plan estimation using multiple 360-images.
• MP3D-FPE-MLV Dataset: A dataset for multiview layout estimation. Particularly, for self-training 360-layout models.
• 360-HM3D-MLV Dataset: A dataset for multiview layout estimation using HM3D dataset.

Installation

Create a virtual environment

conda create -n mv_datasets python=3.9
conda activate mv_datasets

Install the package from the repository

# Installing version v1.0.24.11.24
pip install git+https://github.com/EnriqueSolarte/multiview-datasets.git@v1.0.24.11.24

For installing this package in dev mode (for development)

git clone git@github.com:EnriqueSolarte/multiview-datasets.git
cd multiview-datasets
pip install -e .

Download MLV-Datasets

The mvl-datasets is host in huggingface/EnriqueSolarte/mvl_datasets 🤗.

Warning

To access to the mvl-datasets (i.e., hm3d-mvl, mp3d-mvl, and zind-mvl), you need to create an account and login on HuggingFace and accept the terms and conditions described HERE.

After you get access to the datasets, you have to login your account in your system by following the next commands:

# Install huggingface CLI 
pip install -U "huggingface_hub[cli]"

# Login to your account
huggingface-cli login

# Login to your account
huggingface-cli whoami

Ideally, after login you can download the mvl-dataset by executing the next commands:

# To download and save the dataset in DIR_DATASET
python examples/download_mvl_data/mvl_datasets.py dir_mvl_dataset=${DIR_DATASET}

# To download mp3d_fpe_mvl, hm3d_mvl, or zind_mvl 
python examples/download_mvl_data/mvl_datasets.py dir_mvl_dataset=${DIR_DATASET} dataset=${DATASET_NAME}

# To use the default OmegaConfig in the examples/download_mvl_data/cfg.yaml 
python examples/download_mvl_data/mvl_datasets.py

After you download the dataset, you can visualize and check it as follows:

python examples/visualizations/vis_2d_boundaries.py dir_datasets=${DIR_DATASET}

python examples/visualizations/vis_3d_boundaries.py dir_datasets=${DIR_DATASET}

vis_2d_boundaries.py	vis_3d_boundaries.py

Note

These datasets were officially proposed in 360-MLC - NeuriPS'22 and Ray-casting MLC - ECCV'24. If you use them, please cite them accordingly.

For the hm3d_mvl dataset please cite the following paper:

@article{solarte2024_ray_casting_mlc,
    title   ={Self-training Room Layout Estimation via Geometry-aware Ray-casting}, 
    author  ={Bolivar Solarte and Chin-Hsuan Wu and Jin-Cheng Jhang and Jonathan Lee and Yi-Hsuan Tsai and Min Sun},
    journal ={European Conference on Computer Vision (ECCV)},
    year    ={2024},
    url     ={https://arxiv.org/abs/2407.15041}, 
}

For the mp3d_fpe_mvl dataset please cite the following paper:

@article{Solarte2022_360_MLC,
    title   ={360-mlc: Multi-view layout consistency for self-training and hyper-parameter tuning},
    author  ={Solarte, Bolivar and Wu, Chin-Hsuan and Liu, Yueh-Cheng and Tsai, Yi-Hsuan and Sun, Min},
    journal ={Advances in Neural Information Processing Systems (NeurIPS)},
    volume  ={35},
    pages   ={6133--6146},
    year    ={2022}
}

For the zind-mvl dataset please cite the following paper:

@inproceedings{ZInD,
  title     = {Zillow Indoor Dataset: Annotated Floor Plans With 360º Panoramas and 3D Room Layouts},
  author    = {Cruz, Steve and Hutchcroft, Will and Li, Yuguang and Khosravan, Naji and Boyadzhiev, Ivaylo and Kang, Sing Bing},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2021},
  pages     = {2133--2143}
}