Deep Reinforcement Learning for Identifying Metastable Catalysts

Deep reinforcement learning for predicting kinetic pathways to surface reconstruction in a ternary alloy
Junwoong Yoon*, Zhonglin Cao*, Rajesh Raju*, Yuyang Wang, Robert Burnley, Andrew J. Gellman, Amir Barati Farimani⁺, Zachary W. Ulissi⁺
(*equal contribution, ⁺corresponding authors)
Carnegie Mellon University

[PDF]

If you find this work useful in your research, please cite:

@article{yoon2021deep,
    title={Deep reinforcement learning for predicting kinetic pathways to surface reconstruction in a ternary alloy},
    author={Yoon, Junwoong and Cao, Zhonglin and Raju, Rajesh and Wang, Yuyang and Burnley, Robert and Gellman, Andrew J and Farimani, Amir Barati and Ulissi, Zachary W},
    journal={Machine Learning: Science and Technology},
    year={2021},
    publisher={IOP Publishing}
}

Installation

Custom Installation

1. Clone the github repo

$ git clone https://github.com/ulissigroup/catgym
$ cd catgym
$ conda env create --name catgym
$ conda activate catgym
$ conda install -c anaconda pip

2. Install Sella

$ pip install numpy
$ pip install git+https://github.com/ulissigroup/sella.git

3. Install amptorch

$ pip install git+https://github.com/ulissigroup/amptorch.git
$ pip install cffi pyyaml tqdm braceexpand
$ pip install --no-deps git+https://github.com/MDIL-SNU/SIMPLE-NN.git

4. Install other packages

$ python setup.py develop

Docker Installation

$ git clone https://github.com/ulissigroup/catgym
$ cd catgym
$ docker run --rm -p 8888:8888 --gpus all -e JUPYTER_ENABLE_LAB=yes -e MKL_NUM_THREADS=1 -e OMP_NUM_THREADS=1 -e NUMEXPR_NUM_THREADS=1 -e MKL_DEBUG_CPU_TYPE=5 -v "$PWD":/home/jovyan/catgym ulissigroup/catgym

• Password for JupyterLab is asdf
• To rebuild the image

$ cd docker
$ docker build . -t catgym

Notes for nersc intallation

• The same conda installation won't work for CPU and GPU and nodes. Something about ASAP3 calculator.
• ffmpeg from conda has a problem when running in parallel. Best to use module load ffmpeg.

Environment

To use the gym-based surface segragation environment, please refer to surface_seg/envs/catgym_env.py.

Training

To train a DRL agent for finding surface segregation trajectories. Please run the file examples/notebooks/Train_CatGym.ipynb.

Pretrained agent

A TRPO agent is trained with the CatGYM environment for 10000 episodes. The trained agent is saved in surface_seg/pretrained folder. Please run the file examples/notebooks/Evaluation.ipynb to find surface reconstruction pathways using the pretrained agent.

Results

(a) An example energy pathway to a global minimum developed by DRl method. Each data point represents the relative energy $\Delta E$ of an Ni-Pd-Au configuration generated after taking a certain action at each timestep. (b) A minimum energy pathway, created by NEB, to the same global minimum. (c) Ni(green)-Pd(blue)-Au(gold) configurations of the initial state, transitions states built by DRL method and NEB, and the global minimum.

Acknowledgement

The information, data, or work presented herein was funded in part by the Advanced Research Projects Agency – Energy (ARPA-E), U.S. Department of Energy, under Award Number DE-AR0001221.