PoseStylizer

PyTorch implementation of "Generating Person Images with Appearance-aware Pose Stylizer" [IJCAI 2020].

@inproceedings{huang2020generating,
  title={Generating Person Images with Appearance-aware Pose Stylizer},
  author={Huang, Siyu and Xiong, Haoyi and Cheng, Zhi-Qi and Wang, Qingzhong 
          and Zhou, Xingran and Wen, Bihan and Huan, Jun and Dou, Dejing},
  booktitle={IJCAI},
  year={2020}
}

Getting Started

Requirement

• Python 3
• PyTorch 1.0.1

Installation

• Clone this repo:

git clone https://github.com/siyuhuang/PoseStylizer.git
cd PoseStylizer

Data Preperation

1. Download the Market-1501 dataset dataset/market_data.zip and the DeepFashion dataset dataset/fashion_data.zip from Google Drive / Baidu Disk (Password: jl0s). The zip files include images /train /test, keypoint annotations annotation.csv, and pose transfer pairs pairs.csv.

cd dataset
unzip market_data.zip
unzip fashion_data.zip
cd ..

2. Generate the pose heatmaps. Note, the disk space of generated heatmaps are extremely large (~18GB for Market-1501 and ~160GB for DeepFashion).

python tool/generate_pose_map_market.py
python tool/generate_pose_map_fashion.py

Test with Pretrained Models

Download our pretrained checkpoints from Google Drive / Baidu Disk (Password: jl0s).

• Market-1501

bash test_market.sh

• DeepFashion

bash test_fashion.sh

Training

• Market-1501

bash train_market.sh

• DeepFashion

bash train_fashion.sh

Note: We use 8 GPUs for training by default. If you have less GPUs, change --gpu_ids and --batchSize accordingly. The results are competitive to the results in our paper.

Evaluation

SSIM, IS, L1, mask-SSIM, mask-IS, mask-L1

• Tensorflow 1.14.1 (Python3) is required.

• Market-1501

python tool/getMetrics_market.py

• DeepFashion

python tool/getMetrics_fashion.py

PCKh

• Download OpenPose pose estimator from Google Drive / Baidu Disk (Password: jl0s). Put pose_estimator.h5 under the root folder PoseStylizer.
• Tensorflow 1.14.1 (Python2) is required.

1. Crop generated images from /results folder.

python tool/crop_market.py

    or

python tool/crop_fashion.py

2. Compute keypoints coordinates. Change the paths input_folder and output_path in tool/compute_coordinates.py.

python2 tool/compute_coordinates.py

3. Compute PCKh score.

python tool/calPCKH_market.py

    or

python tool/calPCKH_fashion.py

Acknowledgments

The code is written based on nice frameworks pytorch-CycleGAN-and-pix2pix and Pose-Transfer. The code is written by Dr. Siyu Huang.