-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
880ec28
commit 8247bf4
Showing
1 changed file
with
58 additions
and
40 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,59 +1,77 @@ | ||
| # BrainLens | ||
| <a name="readme-top"></a> | ||
|
|
||
| * DVC | ||
| * Python3 and pip | ||
| * Access to IBM Cloud Object Storage | ||
|
|
||
| ## 🏃🏻 Running Project | ||
| <!-- PROJECT SHIELDS --> | ||
| [![Contributors][contributors-shield]][contributors-url] | ||
| [![Forks][forks-shield]][forks-url] | ||
| [![Stargazers][stars-shield]][stars-url] | ||
| [![Issues][issues-shield]][issues-url] | ||
| [![MIT License][license-shield]][license-url] | ||
| [![LinkedIn][linkedin-shield]][linkedin-url] | ||
|
|
||
| ### 🔑 Setup IBM Bucket Credentials for IBM COS | ||
|
|
||
| #### MacOS and Linux | ||
| Setup your credentials on ```~/.aws/credentials``` and ```~/.aws/config```. DVC works perfectly with IBM Obejct Storage, although it uses S3 protocol, you can also see this in other portions of the repository. | ||
|
|
||
| <!-- PROJECT LOGO --> | ||
| <br /> | ||
| <div align="center"> | ||
|
|
||
| ~/.aws/credentials | ||
| <h3 align="center">BrainLens: Image Reconstruction from Brain fMRI imaging using Diffusion Models</h3> | ||
|
|
||
| ```credentials | ||
| [default] | ||
| aws_access_key_id = {Key ID} | ||
| aws_secret_access_key = {Access Key} | ||
| ``` | ||
| <p align="center"> | ||
| <!-- <a href="https://github.com/ahmed-alllam/BrainLens">View Demo</a> --> | ||
| <!-- · --> | ||
| <a href="https://github.com/ahmed-alllam/BrainLens/issues">Report Bug</a> | ||
| · | ||
| <a href="https://github.com/ahmed-alllam/BrainLens/issues">Request Feature</a> | ||
| </p> | ||
| </div> | ||
|
|
||
|
|
||
| ### ✅ Pre-commit Testings | ||
| <!-- ABOUT THE PROJECT --> | ||
|
|
||
| In order to activate pre-commit testing you need ```pre-commit``` | ||
|
|
||
| Installing pre-commit with pip | ||
| ``` | ||
| pip install pre-commit | ||
| ``` | ||
| ## About The Project | ||
|
|
||
| Installing pre-commit on your local repository. Keep in mind this creates a Github Hook. | ||
| ``` | ||
| pre-commit install | ||
| ``` | ||
| This project aims to reconstruct visual images from brain fMRI data using diffusion models. The project is inspired by the paper [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://arxiv.org/abs/2305.18274). | ||
|
|
||
| Now everytime you make a commit, it will run some tests defined on ```.pre-commit-config.yaml``` before allowing your commit. | ||
| The basic idea of the project is to train a model to reconstruct images from fMRI data. The model is split into two main pipelines, one to capute higher-level semantic features of the image from fMRI data, and the other one to capture the lower-level details of the image. The first pipeline is trained using contrastive learning to make the latent representation of the fMRI similar to the CLIP embedding of the image, and thus the model is able to capture the semantic features of the image. The second pipeline is trained using diffusion autoencoder models to capture the lower-level details of the image. The two pipelines are then combined to reconstruct the image from the fMRI data using Versatile Diffusion model. | ||
|
|
||
| **Example** | ||
| ``` | ||
| $ git commit -m "Example commit" | ||
| > **Note** | ||
| > This project is still under development and is not yet ready for deployment, but feel free to fork it and use it for your own purposes! | ||
| black....................................................................Passed | ||
| pytest-check.............................................................Passed | ||
| ``` | ||
| ## Data Source | ||
|
|
||
| The dataset used in this project is from the Natural Scenes Dataset (NSD), which is a large-scale fMRI dataset conducted at ultra-high-field (7T) strength at the Center of Magnetic Resonance Research (CMRR) at the University of Minnesota. The dataset consists of whole-brain, high-resolution fMRI measurements of 8 healthy adult subjects while they viewed thousands of color natural scenes over the course of 30–40 scan sessions. While viewing these images, subjects were engaged in a continuous recognition task in which they reported whether they had seen each given image at any point in the experiment. [^1^] | ||
|
|
||
| ### ⚗️ Using DVC | ||
| ## Tech Stack | ||
|
|
||
| Download data from the DVC repository(analog to ```git pull```) | ||
| ``` | ||
| dvc pull | ||
| ``` | ||
| The project utilizes the following technologies and libraries: | ||
|
|
||
| Reproduces the pipeline using DVC | ||
| ``` | ||
| dvc repro | ||
| ``` | ||
| -  | ||
| -  | ||
| -  | ||
| -  | ||
| -  | ||
| -  | ||
|
|
||
|
|
||
| ### References | ||
|
|
||
| [^1^]: [Natural Scenes Dataset (NSD)](https://naturalscenesdataset.org/) | ||
|
|
||
|
|
||
|
|
||
| <!-- MARKDOWN LINKS & IMAGES --> | ||
| [contributors-shield]: https://img.shields.io/github/contributors/ahmed-alllam/BrainLens.svg?style=for-the-badge | ||
| [contributors-url]: https://github.com/ahmed-alllam/BrainLens/graphs/contributors | ||
| [forks-shield]: https://img.shields.io/github/forks/ahmed-alllam/BrainLens.svg?style=for-the-badge | ||
| [forks-url]: https://github.com/ahmed-alllam/BrainLens/network/members | ||
| [stars-shield]: https://img.shields.io/github/stars/ahmed-alllam/BrainLens.svg?style=for-the-badge | ||
| [stars-url]: https://github.com/ahmed-alllam/BrainLens/stargazers | ||
| [issues-shield]: https://img.shields.io/github/issues/ahmed-alllam/BrainLens.svg?style=for-the-badge | ||
| [issues-url]: https://github.com/ahmed-alllam/BrainLens/issues | ||
| [license-shield]: https://img.shields.io/github/license/ahmed-alllam/BrainLens.svg?style=for-the-badge | ||
| [license-url]: https://github.com/ahmed-alllam/BrainLens/blob/master/LICENSE.txt | ||
| [linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555 | ||
| [linkedin-url]: https://linkedin.com/in/ahmed-e-allam | ||
| [product-screenshot]: images/screenshot.png |