Disentangled-Person-Image-Generation

Tensorflow implementation of CVPR 2018 paper Disentangled Person Image Generation

Two stages framework

Stage-I: reconstruction

Sampling phase

Dependencies

• python 2.7
• tensorflow-gpu (1.4.1)
• numpy (1.14.0)
• Pillow (5.0.0)
• scikit-image (0.13.0)
• scipy (1.0.1)
• matplotlib (2.0.0)

Resources

• Pretrained models: Market-1501, DeepFashion.
• Training data in tf-record format: Market-1501, DeepFashion.
• Testing data in tf-record format: Market-1501, DeepFashion.
• Raw data: Market-1501, DeepFashion
• Virtual Market Dataset with 500 ID x 24 images: VirtualMarket

TF-record data preparation steps

You can skip this data preparation procedure if directly using the tf-record data files.

1. cd datasets
2. ./run_convert_market.sh to download and convert the original images, poses, attributes, segmentations
3. ./run_convert_DF.sh to download and convert the original images, poses
4. [Optional] ./run_convert_RCV.sh to convert the original images and pose coordinates, i.e. (row, column, visibility) (e.g. from OpenPose or MaskRCNN), which can be useful for other datasets. Note: we also provide the convert code for Market-1501 Attribute and Market-1501 Segmentation results from PSPNet. These extra info. are provided for further research. In our experiments, pose mask are ontained from pose key-points (see _getPoseMask function in convert .py files).

Training steps

1. Download the tf-record training data.
2. Modify the log_dir and log_dir_pretrain in the run_market_train.sh/run_DF_train.sh scripts.
3. run run_market_train.sh/run_DF_train.sh

Note: we use a triplet instead of pair real/fake for adversarial training to keep training more stable.

Testing steps

1. Download the pretrained models and tf-record testing data.
2. Modify the log_dir and log_dir_pretrain in the run_market_test.sh/run_DF_test.sh scripts.
3. run run_market_test.sh/run_DF_test.sh

Fg/Bg/Pose sampling on Market-1501

Appearance sampling on DeepFashion dataset

Pose sampling on DeepFashion dataset

Pose interpolation between real images

• Between same person:
• Between different persons:

Pose guided person image generation

Citation

@inproceedings{ma2018disentangled,
  title={Disentangled Person Image Generation},
  author={Ma, Liqian and Sun, Qianru and Georgoulis, Stamatios and Van Gool, Luc and Schiele, Bernt and Fritz, Mario},
  booktitle={{IEEE} Conference on Computer Vision and Pattern Recognition},
  year={2018}
}

Related projects

• Pose-Guided-Person-Image-Generation
• BEGAN-tensorflow
• improved_wgan_training