Table of Contents:

• Maui Simulation Setup for IT4Innovations
• Quick Start Guide
• Interactive Plots
• Paper and Citation
• Contact
• Acknowledgements
• License

Maui Simulation Setup for IT4Innovations

This repository contains the setup of the Maui Scheduler to simulate the job scheduling of IT4Innovations' large-scale PBS based Salomon cluster. It has been used in our paper Job Simulation for large-scale PBS based Clusters with the Maui Scheduler (see section Paper and Citation below) and is made available to enable other PBS based computing centers to run simulations. The following files are available:

• Dockerfile: To create a Docker image with the Maui Scheduler environment. It includes PBS for satisfying the dependencies only, which does not need to run as a service.
• maui.diff: Modifications applied to the Maui Scheduler for our simulation runs.
• maui.cfg: Configuration file for the Maui Scheduler.
• Resource.Trace1: Trace file for Salomon cluster resources.
• Workload.Trace_example: Workload trace file (example only).
• control_example.sh: Control script for the simulation (example only).

Note: No real workloads are made available due to privacy reasons.

Quick Start Guide

1. Build the Docker image:
   Change the user name (replace userx) and paths in maui.cfg and Dockerfile to reflect your environment. Then, build the Docker image:
   $ docker build -t maui_simulator .

2. Start the Docker container:
   $ docker run --rm -it --hostname mauisim -v $PWD:/home/userx/workdir maui_simulator
   Change the path /home/userx/workdir according to the previous changes.

3. Start simulation:
   Within the Docker container, start the Maui Scheduler's simulator:
   /usr/local/maui/sbin/maui --host=mauisim &
   The default port Maui receives commands is 40559. A different one can be specified by adding the option --port=XYZ. Then, execute the control script control_example.sh as shown in its header comment. Alternatively you can manually use the Maui Scheduler tools to control the simulation (showstats, schedctl, setres, etc.).
   The results can be found in the stats directory (file simstat.out).

Please note that /usr/local/maui/maui.cfg is used by default. It is a link to /home/userx/workdir/maui.cfg (with userx set to the user you defined in the Dockerfile). If you replace the maui.cfg make sure to either use the same target of the link or change the link in the Dockerfile. Alternatively you can start Maui with /usr/local/maui/sbin/maui --host=mauisim --configfile=your_maui.cfg.

Interactive Plots

You can find the plots from the paper as a Jupyter notebook file (interactive_plots/plots.ipynb). It loads the plots from Python pickle files (*.pkl). Open the file in your own Jupyter notebook environment for interactive display.

You can get a preview (static only) using the Jupyter nbviewer:

Note:
To avoid incompatibility, please use the same Jupyter notebook environment we have used for creating the pickle files (uses Matplotlib 2.2.x). You can use the Jupyter notebook scipy-notebook (tag: 265297f221de) from Docker Hub. In the subfolder interactive_plots of our repository, execute the following command:
$ docker run --rm -p 8888:8888 -v "$PWD":/home/jovyan/interactive_plots jupyter/scipy-notebook:265297f221de
Once prompted, open the printed URL in the browser. Open the Jupyter notebook file mentioned above and execute the notebook cell for interactive plots.

Example of an interactive plot:

Paper and Citation

The original paper is available for download on iadis digital library.

@conference{Zitzlsberger20181,
    author={Zitzlsberger, G. and Jansík, B. and Martinovič, J.},
    title={Job simulation for large-scale PBS based clusters with the Maui Scheduler},
    journal={MCCSIS 2018 - Multi Conference on Computer Science and Information Systems; Proceedings of the International Conferences on Big Data Analytics, Data Mining and Computational Intelligence 2018, Theory and Practice in Modern Computing 2018 and Connected Smart Cities 2018},
    year={2018},
    pages={137-145}
}

The extended version of the paper, published by the journal IADIS INTERNATIONAL JOURNAL ON COMPUTER SCIENCE AND INFORMATION SYSTEMS Vol.13 Number 2 (2018), is available for download on iadis digital library.

@article{Zitzlsberger20182,
    author={Zitzlsberger, G. and Jansík, B. and Martinovič, J.},
    title={Feasibility analysis of using the Maui Scheduler for job simulation of large-scale PBS based clusters},
    journal={IADIS International Journal on Computer Science and Information Systems},
    volume={13},
    number={2},
    year={2018},
    pages={47--61}
}

Contact

Should you have any feedback or questions, please contact the author: Georg Zitzlsberger (georg.zitzlsberger(a)vsb.cz).

Acknowledgments

This work was supported by The Ministry of Education, Youth and Sports from the Large Infrastructures for Research, Experimental Development and Innovations project ”IT4Innovations National Supercomputing Center – LM2015070”, by the ERDF in the IT4Innovations national supercomputing center – path to exascale project (CZ.02.1.01/0.0/0.0/16_013/0001791) within the OPRDE, and ANTAREX, a project supported by the EU H2020 FET-HPC program under grant agreement No. 671623.

License

This project is made available under the BSD 2-Clause License.