-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit ae4f308
Showing
188 changed files
with
17,258 additions
and
0 deletions.
There are no files selected for viewing
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 297f818221c985ef6594bc5c8dc4dd09 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
.. Phoebus documentation master file, created by | ||
sphinx-quickstart on Thu Sep 26 15:18:20 2024. | ||
You can adapt this file completely to your liking, but it should at least | ||
contain the root `toctree` directive. | ||
Phoebus: Performance portable GRRMHD for supernovae, mergers, and more | ||
====================================================================== | ||
|
||
`Phoebus`_ is a performance portable general relativistic neutrino radiation magnetohydrodynamics code built upon the `Parthenon`_ adaptive mesh refinement framework. | ||
|
||
.. _Parthenon: https://github.com/parthenon-hpc-lab/parthenon | ||
.. _Phoebus: https://github.com/lanl/phoebus | ||
|
||
Key Features | ||
^^^^^^^^^^^^^ | ||
|
||
* Finite volume GRMHD | ||
* Neutrino transport with Monte Carlo and moment methods | ||
* Analytic and tabulated spacetimes | ||
* Monopolar general relativistic gravity for core-collapse supernovae | ||
* Lagrangian tracer particles | ||
* Support for arbitrary equations of state | ||
|
||
.. note:: | ||
|
||
These docs are under active development. | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
:caption: Contents: | ||
:glob: | ||
|
||
src/* | ||
|
||
Indices and tables | ||
================== | ||
|
||
* :ref:`genindex` | ||
* :ref:`search` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,190 @@ | ||
Building | ||
======== | ||
|
||
Obtaining the Source Code | ||
------------------------- | ||
|
||
``phoebus`` uses submodules extensively. To make sure you get them all, | ||
clone it as | ||
|
||
.. code:: bash | ||
git clone --recursive git@github.com:lanl/phoebus.git | ||
or as | ||
|
||
.. code:: bash | ||
git clone git@github.com:lanl/phoebus.git | ||
cd phoebus | ||
git submodule update --init --recursive | ||
Prerequisites | ||
------------- | ||
|
||
To build ``phoebus``, you need to create a build directly, as in-source builds are not supported. | ||
After cloning the repository, | ||
|
||
.. code:: bash | ||
cd phoebus | ||
mkdir bin | ||
cd bin | ||
Example builds | ||
-------------- | ||
|
||
Below are some example build commands | ||
|
||
MPI-parallel only | ||
~~~~~~~~~~~~~~~~~ | ||
|
||
The following will build ``phoebus`` with MPI parallelism but no shared | ||
memory parallelism | ||
|
||
.. code:: bash | ||
cmake .. | ||
make -j | ||
OpenMP-parallel | ||
~~~~~~~~~~~~~~~ | ||
|
||
The following will build ``phoebus`` with OpenMP parallelism only | ||
|
||
.. code:: bash | ||
cmake -DPHOEBUS_ENABLE_MPI=Off -DPHOEBUS_ENABLE_OPENMP=ON .. | ||
make -j | ||
Cuda | ||
~~~~ | ||
|
||
The following will build ``phoebus`` with no MPI or OpenMP parallelism. | ||
|
||
.. code:: bash | ||
cmake -DPHOEBUS_ENABLE_CUDA=On -DCMAKE_CXX_COMPILER=${HOME}/phoebus/external/singularity-eos/utils/kokkos/bin/nvcc_wrapper -DKokkos_ARCH_HSW=ON -DKokkos_ARCH_VOLTA70=ON -DPHOEBUS_ENABLE_MPI=OFF .. | ||
A few notes for this one: - Note here the ``-DCMAKE_CXX_COMPILER`` flag. | ||
This is necessary. You *must* set the compiler to ``nvcc_wrapper`` | ||
provided by Kokkos. - Note the ``-DKokkos_ARCH_*`` flags. Those set the | ||
host and device microarchitectures and are required. The choice here is | ||
on an ``x86_64`` machine with a ``volta`` GPU. | ||
|
||
Build Options | ||
------------- | ||
|
||
The build options explicitly provided by ``phoebus`` are: | ||
|
||
+-----------------------+----------+-----------------------------------------------+ | ||
| Option | Default | Comment | | ||
+=======================+==========+===============================================+ | ||
| PHOEBUS_ENABLE_CUDA | OFF | Cuda | | ||
+-----------------------+----------+-----------------------------------------------+ | ||
| PHOEBUS_ENABLE_HDF5 | ON | HDF5. Required for output and restarts. | | ||
+-----------------------+----------+-----------------------------------------------+ | ||
| PHOEBUS_ENABLE_MPI | ON | MPI. Required for distributed memory | | ||
| | | parallelism. | | ||
+-----------------------+----------+-----------------------------------------------+ | ||
| PHOEBUS_ENABLE_OPENMP | OFF | OpenMP. Required for shared memory | | ||
| | | parallelism. | | ||
+-----------------------+----------+-----------------------------------------------+ | ||
| MACHINE_CFG | None | Machine-specific config file, optional. | | ||
+-----------------------+----------+-----------------------------------------------+ | ||
|
||
Some relevant settings from Parthenon and Kokkos you may need to play | ||
with are: | ||
|
||
+---------------------+----------+---------------------------------------------+ | ||
| Option | Default | Comment | | ||
+=====================+==========+=============================================+ | ||
| CMAKE_CXX_COMPILER | None | Must be set to ``nvcc_wrapper`` with cuda | | ||
| | | backend | | ||
+---------------------+----------+---------------------------------------------+ | ||
| Kokkos_ARCH_XXXX | OFF | You must set the GPU architecture when | | ||
| | | compiling for Cuda | | ||
+---------------------+----------+---------------------------------------------+ | ||
|
||
You can see all the Parthenon build options | ||
`here <https://github.com/lanl/parthenon/blob/develop/docs/building.md>`__ | ||
and all the Kokkos build options | ||
`here <https://github.com/kokkos/kokkos/wiki/Compiling>`__ | ||
|
||
Cmake machine configs | ||
--------------------- | ||
|
||
If you are proficient with ``cmake`` You can optionally write a | ||
``cmake`` file that sets the configure parameters that you like on a | ||
given machine. Both ``phoebus`` and ``parthenon`` can make use of it. | ||
You can point to the file with | ||
|
||
:: | ||
|
||
-DMACHINE_CFG=path/to/machine/file | ||
|
||
at config time or by setting the environment variable ``MACHINE_CFG`` to | ||
point at it, e.g., | ||
|
||
.. code:: bash | ||
export MACHINE_CFG=path/to/machine/file | ||
An example machine file might look like | ||
|
||
:: | ||
|
||
# Machine file for x86_64-volta on Darwin | ||
message(STATUS "Loading machine configuration for Darwin x86-volta node") | ||
message(STATUS "Assumes: module load module load gcc/7.4.0 cuda/10.2 openmpi/4.0.3-gcc_7.4.0 anaconda/Anaconda3.2019.10 cmake && spack load hdf5") | ||
message(STATUS "Also assumes you have a valid spack installation loaded.") | ||
|
||
set(PHOEBUS_ENABLE_CUDA ON CACHE BOOL "Cuda backend") | ||
set(PHOEBUS_ENABLE_MPI OFF CACHE BOOL "No MPI") | ||
set(Kokkos_ARCH_HSW ON CACHE BOOL "Haswell target") | ||
set(Kokkos_ARCH_VOLTA70 ON CACHE BOOL "volta target") | ||
set(CMAKE_CXX_COMPILER /home/jonahm/phoebus/external/parthenon/external/Kokkos/bin/nvcc_wrapper CACHE STRING "nvcc wrapper") | ||
|
||
you could then configure and compile as | ||
|
||
.. code:: bash | ||
cmake -DMACHINE_CFG=path/to/machine/file .. | ||
make -j | ||
Running | ||
------- | ||
|
||
Run phoebus from the ``build`` directory as | ||
|
||
.. code:: bash | ||
./src/phoebus -i path/to/input/file.pin | ||
The input files are in ``phoebus/inputs/*``. There’s typically one input | ||
file per problem setup file. | ||
|
||
Submodules | ||
---------- | ||
|
||
- ``parthenon`` asynchronous tasking and block-AMR infrastructure | ||
- ``singularity-eos`` provides performance-portable equations of state | ||
and PTE solvers | ||
- ``singularity-opac`` provides performance-portable opacities and | ||
emissivities | ||
- ``Kokkos`` provides performance portable shared-memory parallelism. | ||
It allows our loops to be CUDA, OpenMP, or something else. By default | ||
we use the ``Kokkos`` shipped with ``parthenon``. | ||
|
||
External (Required) | ||
------------------- | ||
|
||
- ``cmake`` for building | ||
|
||
Optional | ||
-------- | ||
|
||
- ``hdf5`` for output (must be parallel if MPI is enabled) | ||
- ``MPI`` for distributed memory parallelism | ||
- ``python3`` for visualization |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
.. _lanlcode: https://lanl.github.io/singularity-eos | ||
|
||
Code of Conduct | ||
===================================== | ||
|
||
We as users, members, contributors, and leaders agree to make participation in our | ||
community a harassment-free experience for everyone. | ||
We will interact in ways that contribute to an open, welcoming, | ||
diverse, inclusive, and healthy community. | ||
|
||
We agree to uphold the `LANL Code of Conduct <lanlcode>`_ in all spaces relevant. | ||
Of note: | ||
|
||
:: | ||
|
||
- Foster a mutually respectful working environment that is free from | ||
discrimination, intimidation, and harassment (including sexual | ||
harassment). | ||
- Do not discriminate against others on the basis of any | ||
characteristic protected by law or LANL policy. | ||
- Do not engage in behaviors that create an offensive, hostile, or | ||
intimidating work environment, including shouting, abusive | ||
language, threats of violence, the use of obscenities or other | ||
non-verbal expressions of aggression, horseplay such as kicking | ||
co-workers’ chairs or using personal insults, or the use of | ||
offensive nicknames or other derogatory remarks. | ||
- Avoid behavior that a reasonable person would find to be demeaning | ||
or humiliating. |
Oops, something went wrong.