Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding neural network inference to NMMA #370

Merged
merged 112 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
07afe91
Update analysis.py
malinadesai May 7, 2024
538b0b6
Update analysis.py
malinadesai May 7, 2024
c2406f9
Create embedding.py
malinadesai May 7, 2024
5c3bf16
Create resnet.py
malinadesai May 7, 2024
94a03f6
Create normalizingflows.py
malinadesai May 7, 2024
e390db3
Create dataprocessing.py
malinadesai May 7, 2024
96874fb
Update analysis.py
malinadesai May 9, 2024
d9b876a
Update analysis.py
malinadesai May 21, 2024
8800fdc
Update analysis.py
malinadesai Jun 25, 2024
8a13004
Update analysis.py
malinadesai Jun 25, 2024
22cb2b1
Update analysis.py
malinadesai Jun 25, 2024
b3d4d79
Update analysis.py
malinadesai Jun 25, 2024
3f030b7
Update analysis.py
malinadesai Jun 25, 2024
b0f0ea4
Update analysis.py
malinadesai Jun 25, 2024
c779fb8
Merge branch 'nuclear-multimessenger-astronomy:main' into main
malinadesai Jun 25, 2024
d06fa0d
Update analysis.py
malinadesai Jun 25, 2024
2e418c2
Update analysis.py
malinadesai Jun 26, 2024
bb7445d
Update normalizingflows.py
malinadesai Jun 26, 2024
5059fbf
Update analysis.py
malinadesai Jun 26, 2024
c6089b1
Update analysis.py
malinadesai Jun 26, 2024
5f91cf2
Update analysis.py
malinadesai Jun 26, 2024
ea33216
Update analysis.py
malinadesai Jun 26, 2024
c362685
Update analysis.py
malinadesai Jun 26, 2024
3239983
Update analysis.py
malinadesai Jun 26, 2024
2c09600
Update analysis.py
malinadesai Jun 26, 2024
c0532e8
Update analysis.py
malinadesai Jun 26, 2024
4a94426
Update analysis.py
malinadesai Jun 26, 2024
43140cb
Update analysis.py
malinadesai Jun 26, 2024
729439a
Update analysis.py
malinadesai Jul 2, 2024
400c2aa
Update analysis.py
malinadesai Jul 2, 2024
9c70086
Update analysis.py
malinadesai Jul 2, 2024
5c5a716
Update analysis.py
malinadesai Jul 2, 2024
9c12b06
Update analysis.py
malinadesai Jul 2, 2024
eed52f6
Update analysis.py
malinadesai Jul 2, 2024
85adf8c
Update analysis.py
malinadesai Jul 2, 2024
306bb41
Update analysis.py
malinadesai Jul 2, 2024
6697925
Update analysis.py
malinadesai Jul 2, 2024
59dd3b9
Update analysis.py
malinadesai Jul 2, 2024
6bb0370
Update analysis.py
malinadesai Jul 2, 2024
32167d9
Update analysis.py
malinadesai Jul 2, 2024
3a4f8ba
Update analysis.py
malinadesai Jul 2, 2024
58271d3
Update analysis.py
malinadesai Jul 2, 2024
43b7f0d
Update analysis.py
malinadesai Jul 2, 2024
848d6aa
Update analysis.py
malinadesai Jul 3, 2024
dd3eef2
Update analysis.py
malinadesai Jul 3, 2024
09519b9
Update analysis.py
malinadesai Jul 3, 2024
a3079c2
Update dataprocessing.py
malinadesai Jul 3, 2024
294c91f
Update analysis.py
malinadesai Jul 3, 2024
4d20ac8
added mlmodel
malinadesai Jul 3, 2024
3f3f2ce
Delete mlmodel directory
malinadesai Jul 3, 2024
9932081
Update analysis.py
malinadesai Jul 3, 2024
b974fcd
Update requirements.txt
malinadesai Jul 3, 2024
d1cecc4
Update requirements.txt
malinadesai Jul 9, 2024
8521b42
Update dataprocessing.py
malinadesai Jul 11, 2024
350aebf
Update analysis.py
malinadesai Jul 11, 2024
9ced67b
Update requirements.txt
malinadesai Jul 11, 2024
4e41c58
Update requirements.txt
malinadesai Jul 11, 2024
2a1fa38
Update requirements.txt
malinadesai Jul 11, 2024
35f9bb3
Update requirements.txt
malinadesai Jul 15, 2024
0f702e9
Update analysis.py
malinadesai Jul 15, 2024
be51d4c
Update analysis.py
malinadesai Jul 15, 2024
27c8ebf
Update analysis.py
malinadesai Jul 15, 2024
975338f
Update analysis.py
malinadesai Jul 15, 2024
87d65ce
Create inference.py
malinadesai Jul 15, 2024
c58e5a7
Update analysis.py
malinadesai Jul 15, 2024
e37b062
Update analysis.py
malinadesai Jul 15, 2024
2f8dcef
Update inference.py
malinadesai Jul 15, 2024
e5bb074
Update inference.py
malinadesai Jul 15, 2024
19eeb17
Update inference.py
malinadesai Jul 15, 2024
dd3fdad
Update analysis.py
malinadesai Jul 15, 2024
cdd4e78
Update dataprocessing.py
malinadesai Jul 15, 2024
7cd2af5
Update embedding.py
malinadesai Jul 15, 2024
4dbfb96
Update normalizingflows.py
malinadesai Jul 15, 2024
c3236f0
Update analysis.py
malinadesai Jul 15, 2024
6c5a0cf
Update analysis.py
malinadesai Jul 15, 2024
95fca35
Update dataprocessing.py
malinadesai Jul 15, 2024
c9ec528
Update analysis.py
malinadesai Jul 15, 2024
ea04bad
Update analysis.py
malinadesai Jul 15, 2024
5ca514f
Add files via upload
malinadesai Jul 15, 2024
7c7d953
Add files via upload
malinadesai Jul 15, 2024
9f21951
Update analysis.py
malinadesai Jul 15, 2024
97f324a
Update analysis.py
malinadesai Jul 15, 2024
f8e908c
Update analysis.py
malinadesai Jul 16, 2024
7b72138
Update analysis.py
malinadesai Jul 16, 2024
0c828d0
Update analysis.py
malinadesai Jul 16, 2024
15a40e2
Update analysis.py
malinadesai Jul 16, 2024
da69a78
Update analysis.py
malinadesai Jul 16, 2024
c4a9ed4
Update analysis.py
malinadesai Jul 16, 2024
ba3aa21
Update analysis.py
malinadesai Jul 16, 2024
c6952d6
Update analysis.py
malinadesai Jul 16, 2024
daddb8c
Update analysis.py
malinadesai Jul 16, 2024
4d5ce61
Update analysis.py
malinadesai Jul 16, 2024
666b2df
Update analysis.py
malinadesai Jul 16, 2024
f32c674
Update analysis.py
malinadesai Jul 16, 2024
b13ac80
Update analysis.py
malinadesai Jul 16, 2024
4444ce8
Update analysis.py
malinadesai Jul 16, 2024
2f3586d
Update analysis.py
malinadesai Jul 16, 2024
f21ee5e
Update analysis.py
malinadesai Jul 16, 2024
27a8f40
Update analysis.py
malinadesai Jul 16, 2024
63d872e
Update analysis.py
malinadesai Jul 16, 2024
a0902c0
Create ml_requirements.txt
malinadesai Jul 17, 2024
a1a6828
Create lfi_analysis.md
malinadesai Jul 17, 2024
79c8112
Update lfi_analysis.md
malinadesai Jul 17, 2024
ec9a748
Update lfi_analysis.md
malinadesai Jul 17, 2024
f19f89e
Update analysis.py
malinadesai Jul 17, 2024
8917470
Update continous_integration.yml
sahiljhawar Jul 17, 2024
747474d
Update continous_integration.yml
sahiljhawar Jul 17, 2024
6351252
Update analysis.py
sahiljhawar Jul 17, 2024
ac84717
add lfi to toctree
sahiljhawar Jul 17, 2024
eee9b22
local_only to False in nn
sahiljhawar Jul 18, 2024
bda86e8
add injection file and injection path
sahiljhawar Jul 18, 2024
4492cd8
update pyproject and add torchvision to req
sahiljhawar Jul 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/continous_integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ jobs:
sudo apt-get update
sudo apt install -y openmpi-bin libopenmpi-dev gfortran build-essential libblas3 libblas-dev liblapack3 liblapack-dev libatlas-base-dev texlive texlive-latex-extra texlive-fonts-recommended dvipng cm-super
python -m pip install --upgrade git+https://github.com/bitranox/wrapt_timeout_decorator.git
python -m pip install pytest pytest-cov flake8 pytest-aiohttp sqlparse freezegun PyJWT joblib tensorflow afterglowpy coveralls
python -m pip install pytest pytest-cov flake8 pytest-aiohttp sqlparse freezegun PyJWT joblib tensorflow afterglowpy coveralls nflows torch torchvision
python -m pip install .
git clone https://github.com/JohannesBuchner/MultiNest && cd MultiNest/build && cmake .. && make && cd ../..
pwd
Expand Down
2 changes: 2 additions & 0 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,8 @@ User Guide
Cluster_Resources
contributing
changelog
lfi_analysis


.. Indices and tables
.. ==================
Expand Down
38 changes: 38 additions & 0 deletions doc/lfi_analysis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Perform Parameter Estimation Using Liklihood Free Inference (LFI)

NMMA is adding machine learning functionality to its currently offered analysis methods. In this initial incorporation, a neural network approach will perform parameter estimation on light curves from BNS events. We will address the limitations first, and then provide an example run.

## Limitations

1. Requires use of the Ka2017 model
2. Requires use of 3 filters, optimally performs best when given ztfg,ztfr,ztfi
3. Requires a time step dT of 0.25

## Functionality

1. Returns a posterior when a light curve is given
2. Returns a posterior when an injection file is given

## Example

### Set up the environment

In addition to installing the standard requirements.txt file, you must also run pip install -r ml_requirments.txt to fulfill the necessary package requirements.

### Generate a simulation set

First, we will create an injection file that describes our light curves. Running the following command line will generate a json file (injection.json). For Ka2017, this will include the parameters: luminosity_distance, timeshift, log10_mej, log10_vej, log10_Xlan, and geocent_time.

nmma_create_injection --prior-file ./priors/Ka2017 --eos-file ./example_files/eos/ALF2.dat --binary-type BNS --filename ./output/injection --n-injection 10 --original-parameters --extension json

We can generate light curves using this injection file with the following command. Here, we can define the start and ending time of the light curve, its filters, and we can add ztf-like noise.

lightcurve-generation --model Ka2017 --outdir outdir --outfile-type json --label test --tmin -2 --tmax 20 --dt 0.25 --filters ztfg,ztfr,ztfi --injection ./outdir/injection.json --injection-detection-limit 22.0,22.0,22.0 --ztf-uncertainties

There are two options when running the analysis. One can run it using the injection file directly, which will cause the analysis to generate a light curve on the fly. Or, the command will accept a pre-generated light curve. To run using LFI, the --sampler argument must be neuralnet. In the first example, we call the injection file and pass the first injection to the analysis. In the second, we take the first light curve generated by the injection file. Both of these commands will output a posterior plot to the provided outdir.

lightcurve-analysis --sampler neuralnet --model Ka2017 --outdir inferences --label with_inj --prior priors/Ka2017.prior --injection outdir/injection.json --injection-num 0 --dt 0.25

lightcurve-analysis --sampler neuralnet --model Ka2017 --outdir inferences --label with_data --data outdir/test_0.json --prior priors/Ka2017.prior


3 changes: 3 additions & 0 deletions ml_requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
torch
nflows
torchvision
Loading
Loading