Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update the requirements and fix the _expand_mask import issue #705

Open
wants to merge 198 commits into
base: main
Choose a base branch
from
Open

update the requirements and fix the _expand_mask import issue #705

wants to merge 198 commits into from

Commits on Sep 16, 2022

  1. Update README.md 

    update instructions for Unified Feature Extraction Interface
    Junnan Li authored Sep 16, 2022
    Configuration menu
    Copy the full SHA
    61e532b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/salesforce/LAVIS

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    8989543 View commit details
    Browse the repository at this point in the history

  3. Update getting_started.rst

    Junnan Li authored Sep 16, 2022
    Configuration menu
    Copy the full SHA
    5498416 View commit details
    Browse the repository at this point in the history

  4. Update getting_started.rst 

    update documentation for feature extraction
    Junnan Li authored Sep 16, 2022
    Configuration menu
    Copy the full SHA
    33a763d View commit details
    Browse the repository at this point in the history

  5. renaming features to embeds_proj.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    f6e1354 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' of https://github.com/salesforce/LAVIS

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    40b5022 View commit details
    Browse the repository at this point in the history

  7. update docs.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    6f6c40e View commit details
    Browse the repository at this point in the history

  8. update docs.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    2e072e6 View commit details
    Browse the repository at this point in the history

  9. update README and docs.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    4e69f40 View commit details
    Browse the repository at this point in the history

  10. update README.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    e4ab2b1 View commit details
    Browse the repository at this point in the history

  11. update README.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    8961e6d View commit details
    Browse the repository at this point in the history

  12. relax hf version.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    f42138b View commit details
    Browse the repository at this point in the history

  13. Update README.md

    @schhoi
    schhoi authored Sep 16, 2022
    Configuration menu
    Copy the full SHA
    b057e38 View commit details
    Browse the repository at this point in the history

  14. Update README.md

    @schhoi
    schhoi authored Sep 16, 2022
    Configuration menu
    Copy the full SHA
    0f94053 View commit details
    Browse the repository at this point in the history

  15. update README.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    5db17b9 View commit details
    Browse the repository at this point in the history

  16. default cache root.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    5d4798a View commit details
    Browse the repository at this point in the history

  17. update README.

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    e28a480 View commit details
    Browse the repository at this point in the history

  18. Merge branch 'main' of https://github.com/salesforce/LAVIS

    @dxli94
    dxli94 committed Sep 16, 2022
    Configuration menu
    Copy the full SHA
    34b3b69 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2022

  1. Update README.md

    Junnan Li authored Sep 19, 2022
    Configuration menu
    Copy the full SHA
    210d9bc View commit details
    Browse the repository at this point in the history

  2. Update README.md 

    fix typos
    Junnan Li authored Sep 19, 2022
    Configuration menu
    Copy the full SHA
    70e5e0f View commit details
    Browse the repository at this point in the history

  3. Update README.md

    @dxli94
    dxli94 authored Sep 19, 2022
    Configuration menu
    Copy the full SHA
    96d474a View commit details
    Browse the repository at this point in the history

  4. update app.

    @dxli94
    dxli94 committed Sep 19, 2022
    Configuration menu
    Copy the full SHA
    5ea4f13 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2022

  1. Update README.md

    @dxli94
    dxli94 authored Sep 20, 2022
    Configuration menu
    Copy the full SHA
    2c1c6dd View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored Sep 20, 2022
    Configuration menu
    Copy the full SHA
    2d13d3c View commit details
    Browse the repository at this point in the history

  3. update docs.

    @dxli94
    dxli94 committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    ef82902 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of https://github.com/salesforce/LAVIS

    @dxli94
    dxli94 committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    3ca8448 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2022

  1. (fix) ALBEF feature extractor mismatch keys. 

    When loading ALBEF feature extractor, it loads pre-trained checkpoints.
Yet the default behavior of load_from_pretrained will rename bert keys.
As a result, weights for ALBEF text encoders are not properly loaded.
This can be confirmed by checking missing keys after loading weights.
This commit fixes the issue by adding an option to not rename the keys.
Other ALBEF task-specific architectures should not be affected.
    @dxli94
    dxli94 committed Sep 22, 2022
    Configuration menu
    Copy the full SHA
    4001f32 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored Sep 22, 2022
    Configuration menu
    Copy the full SHA
    afca26d View commit details
    Browse the repository at this point in the history

  3. remove license headers from create_annotation_12m.ipynb create_annota… 

    …tion_3m.ipynb.
    @dxli94
    dxli94 committed Sep 22, 2022
    Configuration menu
    Copy the full SHA
    1733225 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of https://github.com/salesforce/LAVIS

    @dxli94
    dxli94 committed Sep 22, 2022
    Configuration menu
    Copy the full SHA
    a2cc24f View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2022

  1. Update README.md

    @dxli94
    dxli94 authored Sep 25, 2022
    Configuration menu
    Copy the full SHA
    b964f12 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored Sep 25, 2022
    Configuration menu
    Copy the full SHA
    924d4d4 View commit details
    Browse the repository at this point in the history

  3. Fix LICENSE link

    @kpister
    kpister authored Sep 25, 2022
    Configuration menu
    Copy the full SHA
    aa66440 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2022

  1. Merge pull request salesforce#8 from kpister/patch-1 

    Fix LICENSE link
    @dxli94
    dxli94 authored Sep 27, 2022
    Configuration menu
    Copy the full SHA
    a311bb1 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2022

  1. (feat) update zero-shot classfication demo with ALBEF model.

    @dxli94
    dxli94 committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    130ad3b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/salesforce/LAVIS into main

    @dxli94
    dxli94 committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    0b74b2e View commit details
    Browse the repository at this point in the history

  3. (docs) update docs for ALBEF, BLIP VQA. 

    As suggested by salesforce#16, update comments
to make it clear the number of questions should be equal to number of images.

Also add assertion to ensure this requirment is satisfied.
    @dxli94
    dxli94 committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    80e1c13 View commit details
    Browse the repository at this point in the history

  4. fix: added scikit-image to requirements for run_demo.sh

    @nick-konovalchuk
    nick-konovalchuk committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    8ed6b7a View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2022

  1. Merge pull request salesforce#13 from bottledmind/fix/demo-requirements 

    fix: requirements for run_demo.sh
    @dxli94
    dxli94 authored Oct 7, 2022
    Configuration menu
    Copy the full SHA
    2ba2677 View commit details
    Browse the repository at this point in the history

  2. (feat) add blip itm and tests.

    @dxli94
    dxli94 committed Oct 7, 2022
    Configuration menu
    Copy the full SHA
    40ea7e6 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2022

  1. (docs) add notebook examples.

    @dxli94
    dxli94 committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    b4130cd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/salesforce/LAVIS into main

    @dxli94
    dxli94 committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    a0e98de View commit details
    Browse the repository at this point in the history

  3. Update README.md

    @dxli94
    dxli94 authored Oct 8, 2022
    Configuration menu
    Copy the full SHA
    bd0738a View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2022

  1. Update README.md

    @dxli94
    dxli94 authored Oct 11, 2022
    Configuration menu
    Copy the full SHA
    570aef8 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2022

  1. fix: changed MRO and removed from

    @nick-konovalchuk
    nick-konovalchuk committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    2b592e5 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2022

  1. Merge pull request salesforce#20 from bottledmind/fix/gpt-dialogue-lo… 

    …ading-issue

fix: `gpt_dialogue` loading issue
    @henryhungle
    henryhungle authored Oct 15, 2022
    Configuration menu
    Copy the full SHA
    0fc3171 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2022

  1. (docs) empty readme for PnP-VQA.

    @dxli94
    dxli94 committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    b587543 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored Oct 18, 2022
    Configuration menu
    Copy the full SHA
    a2ed3d4 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2022

  1. (fix) updated SBU annotation url.

    @dxli94
    dxli94 committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    7c5a051 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/salesforce/LAVIS

    @dxli94
    dxli94 committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    c711229 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2022

  1. (fix) upload aokvqa vocab to gcp storage and update urls accordingly.

    @dxli94
    dxli94 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    1daccd8 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2022

  1. Pnp vqa (salesforce#27) 

    Release of PnP-VQA (Anthony TMH et al, EMNLP Findings 2022).

* add cache to gitignore

* pnp-vqa-okvqa

* rename readme

* add extra config and misc change

* edit model path

* add gqa dataset

* add gqa inference for generate

* pnp-vqa misc change

* compute gradcam in batch

* allow straightforward model.predict_answers()

* Created using Colaboratory

* pnp_vqa

* add test and update colab

* add vqav2 test config and sh

* log vqa result

* fix dict key

* reduce memory by offload model for pnpvqa3b

* misc

* update readme
    @anthonytmh
    anthonytmh authored Oct 25, 2022
    Configuration menu
    Copy the full SHA
    28d9d9e View commit details
    Browse the repository at this point in the history

  2. Update README.md to include PnP-VQA.

    @dxli94
    dxli94 authored Oct 25, 2022
    Configuration menu
    Copy the full SHA
    d6c54cf View commit details
    Browse the repository at this point in the history

  3. Update README.md

    @dxli94
    dxli94 authored Oct 25, 2022
    Configuration menu
    Copy the full SHA
    f95a3da View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2022

  1. Update README.md

    @dxli94
    dxli94 authored Oct 28, 2022
    Configuration menu
    Copy the full SHA
    fa8d067 View commit details
    Browse the repository at this point in the history

  2. Fix git url for git clone in documentation (salesforce#21) 

    The git url for git clone in documentation is not working, it's not the Salesforce repository.
    @gschurck
    gschurck authored Oct 28, 2022
    Configuration menu
    Copy the full SHA
    ae75fb6 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2022

  1. bug fix for colab pnp_vqa model_type (salesforce#38)

    @anthonytmh
    anthonytmh authored Nov 7, 2022
    Configuration menu
    Copy the full SHA
    e71f49c View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2022

  1. (fix) downloading sbu caption images from source urls.

    @dxli94
    dxli94 committed Nov 21, 2022
    Configuration menu
    Copy the full SHA
    e6caa1b View commit details
    Browse the repository at this point in the history

  2. (fix) update msrvtt download link, and instructions for further downl… 

    …oads.
    @dxli94
    dxli94 committed Nov 21, 2022
    Configuration menu
    Copy the full SHA
    702c0fc View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2022

  1. Fix error failling VQA evaluation (salesforce#59) 

    Looks good on me. Build passed.
    @raven38
    raven38 authored Dec 8, 2022
    Configuration menu
    Copy the full SHA
    2e97fd0 View commit details
    Browse the repository at this point in the history

  2. Fix barrier on single gpu (salesforce#61) 

    * fix_barrier_on_single_gpu
    @pldlgb
    pldlgb authored Dec 8, 2022
    Configuration menu
    Copy the full SHA
    23d7ca8 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2022

  1. Img2Prompt (salesforce#69) 

    * img2prompt

* fsdf

* final touch before merging img2prompt.

* update README.md

Co-authored-by: jiaxian <2362740862@qq.com>
    @dxli94
    dxli94 and jiaxian authored Dec 21, 2022
    Configuration menu
    Copy the full SHA
    5913ce8 View commit details
    Browse the repository at this point in the history

  2. Update img2prompt_vqa.ipynb

    @dxli94
    dxli94 authored Dec 21, 2022
    Configuration menu
    Copy the full SHA
    7b40786 View commit details
    Browse the repository at this point in the history

  3. Update README.md

    @dxli94
    dxli94 authored Dec 21, 2022
    Configuration menu
    Copy the full SHA
    c61c15d View commit details
    Browse the repository at this point in the history

  4. Update README.md

    @dxli94
    dxli94 authored Dec 21, 2022
    Configuration menu
    Copy the full SHA
    ea65aab View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2022

  1. Update README.md

    Junnan Li authored Dec 22, 2022
    Configuration menu
    Copy the full SHA
    c8dc409 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    Junnan Li authored Dec 22, 2022
    Configuration menu
    Copy the full SHA
    fe50fe4 View commit details
    Browse the repository at this point in the history

  3. Update README.md

    @dxli94
    dxli94 authored Dec 22, 2022
    Configuration menu
    Copy the full SHA
    6e9fd2f View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2023

  1. BLIP-2 integration and amp fix (salesforce#87) 

    * blip2 integration, amp fixing

* update run config

* minor restructure and refactor of blip2 implementations.

* fix cpu device for blip2 example.

* added pyproject.toml

* updates for builds.

* fix salesforce#79.

* added a dev tag for versioning.

Co-authored-by: Junnan Li <“junnan.li@salesforce.com”>
Co-authored-by: Dongxu Li <dongxu_li@outlook.com>
    @dxli94
    3 people authored Jan 26, 2023
    Configuration menu
    Copy the full SHA
    013dd0f View commit details
    Browse the repository at this point in the history

  2. update blip2 instructed generation notebook.

    @dxli94
    dxli94 committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    3ff9616 View commit details
    Browse the repository at this point in the history

  3. update blip2 instructed generation notebook.

    @dxli94
    dxli94 committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    b800408 View commit details
    Browse the repository at this point in the history

  4. update setup.py

    @dxli94
    dxli94 committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    9474e59 View commit details
    Browse the repository at this point in the history

  5. updated requirements.txt with spacy.

    @dxli94
    dxli94 committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    f9e0d44 View commit details
    Browse the repository at this point in the history

  6. fix salesforce#68.

    @dxli94
    dxli94 committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    bda0774 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2023

  1. update blip2 demo

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    229715b View commit details
    Browse the repository at this point in the history

  2. update blip2 demo

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    6f7b3ad View commit details
    Browse the repository at this point in the history

  3. no autocast for cpu

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    48ad4a9 View commit details
    Browse the repository at this point in the history

  4. create blip2 readme

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    7d78e80 View commit details
    Browse the repository at this point in the history

  5. allow cpu autocast for t5

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    192560f View commit details
    Browse the repository at this point in the history

  6. update blip2 demo

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    1bfe81b View commit details
    Browse the repository at this point in the history

  7. update blip2 demo

    root committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    2e958c1 View commit details
    Browse the repository at this point in the history

  8. fix device type for blip2_t5.

    @dxli94
    dxli94 committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    fcc1ad3 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2023

  1. fix autocast on cuda

    root committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    1bd3686 View commit details
    Browse the repository at this point in the history

  2. Revert "fix autocast on cuda" 

    This reverts commit 1bd3686.
    root committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    23145a7 View commit details
    Browse the repository at this point in the history

  3. fix autocast for opt

    root committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    dd1b800 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of github.com:salesforce/LAVIS into main

    root committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    3aa6d64 View commit details
    Browse the repository at this point in the history

  5. fix autocast for opt

    root committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    671f543 View commit details
    Browse the repository at this point in the history

  6. add illustration figure

    Junnan Li authored Jan 28, 2023
    Configuration menu
    Copy the full SHA
    1c48800 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. update eva source

    root committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    561d8af View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:salesforce/LAVIS into main

    root committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    4426e83 View commit details
    Browse the repository at this point in the history

  3. Update README.md

    Junnan Li authored Jan 30, 2023
    Configuration menu
    Copy the full SHA
    d46d598 View commit details
    Browse the repository at this point in the history

  4. update training yaml

    root committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    0d09ec6 View commit details
    Browse the repository at this point in the history

  5. Update README.md

    @dxli94
    dxli94 authored Jan 30, 2023
    Configuration menu
    Copy the full SHA
    30e7970 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2023

  1. Update README.md

    Junnan Li authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    2e96b27 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    Junnan Li authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    b822216 View commit details
    Browse the repository at this point in the history

  3. Update README.md

    Junnan Li authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    9700552 View commit details
    Browse the repository at this point in the history

  4. Update README.md

    @dxli94
    dxli94 authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    a70e108 View commit details
    Browse the repository at this point in the history

  5. Update README.md

    @dxli94
    dxli94 authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    4edf808 View commit details
    Browse the repository at this point in the history

  6. Update README.md

    Junnan Li authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    bbe85c4 View commit details
    Browse the repository at this point in the history

  7. Update README.md

    Junnan Li authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    a852b02 View commit details
    Browse the repository at this point in the history

  8. Update README.md

    Junnan Li authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    f9e02ef View commit details
    Browse the repository at this point in the history

  9. Update README.md

    @dxli94
    dxli94 authored Jan 31, 2023
    Configuration menu
    Copy the full SHA
    c4bdd6b View commit details
    Browse the repository at this point in the history

  10. replace blip2 illustration image with high res.

    @dxli94
    dxli94 committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    458bbc4 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2023

  1. Fixed typo in BLIP2 readme (salesforce#90)

    @chavinlo
    chavinlo authored Feb 1, 2023
    Configuration menu
    Copy the full SHA
    981a636 View commit details
    Browse the repository at this point in the history

  2. Update blip2_qformer.py 

    fix max text length
    @Quan-Sun
    Quan-Sun authored Feb 1, 2023
    Configuration menu
    Copy the full SHA
    24c8811 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2023

  1. Merge pull request salesforce#94 from Quan-Sun/blip2 

    Update blip2_qformer.py to fix max_txt_len
    Junnan Li authored Feb 2, 2023
    Configuration menu
    Copy the full SHA
    1b2766b View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2023

  1. Update README.md

    Junnan Li authored Feb 3, 2023
    Configuration menu
    Copy the full SHA
    d4bd546 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2023

  1. Update README.md

    @dxli94
    dxli94 authored Feb 4, 2023
    Configuration menu
    Copy the full SHA
    5d5a559 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored Feb 4, 2023
    Configuration menu
    Copy the full SHA
    cc598dc View commit details
    Browse the repository at this point in the history

  3. Update README.md

    Junnan Li authored Feb 4, 2023
    Configuration menu
    Copy the full SHA
    392e28f View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2023

  1. update Google Colab link img2_prompt -> blip2 (salesforce#102)

    @mrdbourke
    mrdbourke authored Feb 6, 2023
    Configuration menu
    Copy the full SHA
    fa95939 View commit details
    Browse the repository at this point in the history

  2. Fix salesforce#100. (salesforce#103) 

    Allow using different prompts for blip2_t5 generation. Minor formatting.
    @dxli94
    dxli94 authored Feb 6, 2023
    Configuration menu
    Copy the full SHA
    0769b74 View commit details
    Browse the repository at this point in the history

  3. Added BLIP2 feature extractor and ITM matching (salesforce#104) 

    * add blip2 feature extractor and image-text matching

* minor formatting.

---------

Co-authored-by: root <“junnan.li@salesforce.com”>
    @dxli94
    dxli94 and root authored Feb 6, 2023
    Configuration menu
    Copy the full SHA
    edfd376 View commit details
    Browse the repository at this point in the history

  4. update readme with example notebooks

    Junnan Li authored Feb 6, 2023
    Configuration menu
    Copy the full SHA
    9f6429f View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2023

  1. Add COCO finetuning (salesforce#117) 

    * add scripts for blip2 zero-shot vqa&okvqa evaluation

* delete draft task and add back caption evaluation

* fix amp scaler, fix freeze ViT, add blip-2 finetune script

* remove OKVQA task, apply lemmatization after predict_answers().

* fix optimizer zero_grad under amp

---------

Co-authored-by: root <“junnan.li@salesforce.com”>
    @dxli94
    dxli94 and root authored Feb 8, 2023
    Configuration menu
    Copy the full SHA
    4fede64 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2023

  1. GQA evaluation and BLIP2 ITM bugfix. (salesforce#121) 

    * add scripts for blip2 zero-shot vqa&okvqa evaluation

* delete draft task and add back caption evaluation

* fix amp scaler, fix freeze ViT, add blip-2 finetune script

* remove OKVQA task, apply lemmatization after predict_answers().

* fix optimizer zero_grad under amp

* zero-shot gqa evaluation

* Fix salesforce#119.

---------

Co-authored-by: root <“junnan.li@salesforce.com”>
    @dxli94
    dxli94 and root authored Feb 9, 2023
    Configuration menu
    Copy the full SHA
    1a62e39 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. add BLIP2 pretraining (salesforce#130) 

    * add scripts for blip2 zero-shot vqa&okvqa evaluation

* delete draft task and add back caption evaluation

* fix amp scaler, fix freeze ViT, add blip-2 finetune script

* remove OKVQA task, apply lemmatization after predict_answers().

* fix optimizer zero_grad under amp

* zero-shot gqa evaluation

* Fix salesforce#119.

* update runner - configurable beta and save checkpoint only for requires_grad params

* add blip2 image processor for training

* use blip2 image processor for training

* update blip2 pretraining yaml and add stage-2 pretraining script

* reload checkpoint to cpu first. allow non-strict reloading.

* remove blip2 pretrain.yaml

---------

Co-authored-by: root <“junnan.li@salesforce.com”>
    @dxli94
    dxli94 and root authored Feb 13, 2023
    Configuration menu
    Copy the full SHA
    1118c1b View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2023

  1. blip-2 readme add advice on model selection

    Junnan Li authored Feb 15, 2023
    Configuration menu
    Copy the full SHA
    5b06fa4 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2023

  1. fix max_txt_len typos in blip2_t5 model

    @yqy2001
    yqy2001 committed Feb 19, 2023
    Configuration menu
    Copy the full SHA
    c8d8a8c View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2023

  1. Merge pull request salesforce#141 from yqy2001/main 

    fix max_txt_len typos in blip2_t5 model
    Junnan Li authored Feb 20, 2023
    Configuration menu
    Copy the full SHA
    6958b47 View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2023

  1. pin transformers version >=4.25,<4.27 (salesforce#157) 

    BLIP-2 implementation is not compatible with transformers=4.27, see details salesforce#154. Temporarily pin the version.
    @dxli94
    dxli94 authored Feb 25, 2023
    Configuration menu
    Copy the full SHA
    148f244 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2023

  1. Dev0.5 (salesforce#169) 

    * add BLIP-2 models with CLIP-ViT-large; configurable ca_freq; fix autocast

* use torch.cuda.amp for blip2_t5.

---------

Co-authored-by: root <“junnan.li@salesforce.com”>
    @dxli94
    dxli94 and root authored Mar 3, 2023
    Configuration menu
    Copy the full SHA
    141367f View commit details
    Browse the repository at this point in the history

  2. added clip models vocab file to manifest (salesforce#105)

    @maxi-w
    maxi-w authored Mar 3, 2023
    Configuration menu
    Copy the full SHA
    c8a1735 View commit details
    Browse the repository at this point in the history

  3. Fix command path in docs (salesforce#109) 

    * fix command path

* fix bash -> python
    Taichi Nishimura authored Mar 3, 2023
    Configuration menu
    Copy the full SHA
    dbea3cd View commit details
    Browse the repository at this point in the history

  4. type in getting_started.rst (salesforce#53) 

    devce -> device
    @altryne
    altryne authored Mar 3, 2023
    Configuration menu
    Copy the full SHA
    bb5535b View commit details
    Browse the repository at this point in the history

  5. Fix typo in intro.rst (salesforce#70) 

    accomodate -> accommodate
    @eltociear
    eltociear authored Mar 3, 2023
    Configuration menu
    Copy the full SHA
    b67358e View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2023

  1. Fix BLIP2 mixed precision on CPU (salesforce#179) 

    * allow both cpu string and torch.device to be identified for model loading.

* blip2 amp cpu compatibility.

* use dtype=float16 by default.
    @dxli94
    dxli94 authored Mar 6, 2023
    Configuration menu
    Copy the full SHA
    c6fb2c4 View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2023

  1. Update README.md

    Junnan Li authored Mar 23, 2023
    Configuration menu
    Copy the full SHA
    c5ebfdf View commit details
    Browse the repository at this point in the history

  2. Update README.md

    Junnan Li authored Mar 23, 2023
    Configuration menu
    Copy the full SHA
    f2ec44d View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2023

  1. Renaming img2prompt to img2llm (salesforce#213) 

    * update img2prompt_vqa to img2llm_vqa.

* fix more.
    @dxli94
    dxli94 authored Mar 24, 2023
    Configuration menu
    Copy the full SHA
    745ba73 View commit details
    Browse the repository at this point in the history

  2. 20230224 img2llm (salesforce#214) 

    * update img2prompt_vqa to img2llm_vqa.

* fix more.

* update readme.
    @dxli94
    dxli94 authored Mar 24, 2023
    Configuration menu
    Copy the full SHA
    3a3018e View commit details
    Browse the repository at this point in the history

  3. add back img2prompt README. (salesforce#215)

    @dxli94
    dxli94 authored Mar 24, 2023
    Configuration menu
    Copy the full SHA
    dc34267 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2023

  1. Add HF mention and example code (salesforce#220)

    @NielsRogge
    NielsRogge authored Mar 29, 2023
    Configuration menu
    Copy the full SHA
    f713f5f View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. add blip2 model card

    @LiJunnan1992
    LiJunnan1992 authored Apr 4, 2023
    Configuration menu
    Copy the full SHA
    fd50693 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. instruct models

    root committed May 5, 2023
    Configuration menu
    Copy the full SHA
    e3fd194 View commit details
    Browse the repository at this point in the history

  2. taxi image

    root committed May 5, 2023
    Configuration menu
    Copy the full SHA
    a7af4ae View commit details
    Browse the repository at this point in the history

  3. update blip2-opt for transformers>=4.27

    root committed May 5, 2023
    Configuration menu
    Copy the full SHA
    8ee3c44 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2023

  1. update blip2_opt to work with transformers>=4.27

    root committed May 6, 2023
    Configuration menu
    Copy the full SHA
    62302d1 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2023

  1. transformer version check for blip and blip2 models

    root committed May 7, 2023
    Configuration menu
    Copy the full SHA
    2ee6793 View commit details
    Browse the repository at this point in the history

  2. transformer version check for blip and blip2 models

    root committed May 7, 2023
    Configuration menu
    Copy the full SHA
    1424a68 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2023

  1. support pretrain from scratch

    root committed May 8, 2023
    Configuration menu
    Copy the full SHA
    d771c40 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    Junnan Li authored May 8, 2023
    Configuration menu
    Copy the full SHA
    288c795 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. update example image

    root committed May 9, 2023
    Configuration menu
    Copy the full SHA
    ad83e3e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:salesforce/LAVIS into main

    root committed May 9, 2023
    Configuration menu
    Copy the full SHA
    eaad672 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2023

  1. instructblip

    root committed May 10, 2023
    Configuration menu
    Copy the full SHA
    8aa38dc View commit details
    Browse the repository at this point in the history

  2. Create README.md

    Junnan Li authored May 10, 2023
    Configuration menu
    Copy the full SHA
    cdeff33 View commit details
    Browse the repository at this point in the history

  3. Add files via upload

    @LiJunnan1992
    LiJunnan1992 authored May 10, 2023
    Configuration menu
    Copy the full SHA
    a7236c0 View commit details
    Browse the repository at this point in the history

  4. Update README.md

    Junnan Li authored May 10, 2023
    Configuration menu
    Copy the full SHA
    3a94824 View commit details
    Browse the repository at this point in the history

  5. Update README.md

    Junnan Li authored May 10, 2023
    Configuration menu
    Copy the full SHA
    db854af View commit details
    Browse the repository at this point in the history

  6. Update README.md

    Junnan Li authored May 10, 2023
    Configuration menu
    Copy the full SHA
    06d6ebd View commit details
    Browse the repository at this point in the history

  7. Update README.md

    Junnan Li authored May 10, 2023
    Configuration menu
    Copy the full SHA
    627adbe View commit details
    Browse the repository at this point in the history

  8. Update README.md

    Junnan Li authored May 10, 2023
    Configuration menu
    Copy the full SHA
    9e66acc View commit details
    Browse the repository at this point in the history

Commits on May 11, 2023

  1. Add files via upload

    @LiJunnan1992
    LiJunnan1992 authored May 11, 2023
    Configuration menu
    Copy the full SHA
    afb3338 View commit details
    Browse the repository at this point in the history

  2. Add files via upload

    @LiJunnan1992
    LiJunnan1992 authored May 11, 2023
    Configuration menu
    Copy the full SHA
    2eda702 View commit details
    Browse the repository at this point in the history

  3. Delete showcase.png

    Junnan Li authored May 11, 2023
    Configuration menu
    Copy the full SHA
    1a9aa88 View commit details
    Browse the repository at this point in the history

  4. Add files via upload

    @LiJunnan1992
    LiJunnan1992 authored May 11, 2023
    Configuration menu
    Copy the full SHA
    fd83234 View commit details
    Browse the repository at this point in the history

  5. Update README.md

    Junnan Li authored May 11, 2023
    Configuration menu
    Copy the full SHA
    11c771f View commit details
    Browse the repository at this point in the history

  6. Update README.md

    Junnan Li authored May 11, 2023
    Configuration menu
    Copy the full SHA
    e075e97 View commit details
    Browse the repository at this point in the history

  7. Update README.md

    Junnan Li authored May 11, 2023
    Configuration menu
    Copy the full SHA
    52776dd View commit details
    Browse the repository at this point in the history

Commits on May 12, 2023

  1. Update README.md

    Junnan Li authored May 12, 2023
    Configuration menu
    Copy the full SHA
    ecf9ff2 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    Junnan Li authored May 12, 2023
    Configuration menu
    Copy the full SHA
    e4bd4fd View commit details
    Browse the repository at this point in the history

  3. Update README.md

    Junnan Li authored May 12, 2023
    Configuration menu
    Copy the full SHA
    6bd0b9f View commit details
    Browse the repository at this point in the history

Commits on May 15, 2023

  1. fix blip2 clip vit layer

    root committed May 15, 2023
    Configuration menu
    Copy the full SHA
    43cd66f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:salesforce/LAVIS into main

    root committed May 15, 2023
    Configuration menu
    Copy the full SHA
    3bd5a7d View commit details
    Browse the repository at this point in the history

Commits on May 16, 2023

  1. Update README.md

    Junnan Li authored May 16, 2023
    Configuration menu
    Copy the full SHA
    f8db665 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2023

  1. Update clip_vit.py

    Junnan Li authored May 18, 2023
    Configuration menu
    Copy the full SHA
    fa300eb View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. fix loss logging issue

    root committed May 23, 2023
    Configuration menu
    Copy the full SHA
    af610d2 View commit details
    Browse the repository at this point in the history

  2. Blip diffusion first cut (salesforce#319) 

    * added an empty project page for blip-diffusion.

* add blip-diffusion teaser image.

* enlarge teaser image.
    @dxli94
    dxli94 authored May 23, 2023
    Configuration menu
    Copy the full SHA
    6f5a165 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2023

  1. Update README.md

    @dxli94
    dxli94 authored May 24, 2023
    Configuration menu
    Copy the full SHA
    b3a3fbf View commit details
    Browse the repository at this point in the history

Commits on May 25, 2023

  1. Update README.md

    @dxli94
    dxli94 authored May 25, 2023
    Configuration menu
    Copy the full SHA
    b9f9e97 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored May 25, 2023
    Configuration menu
    Copy the full SHA
    3d2b535 View commit details
    Browse the repository at this point in the history

  3. fix gqa post-processing

    root committed May 25, 2023
    Configuration menu
    Copy the full SHA
    7a08d0c View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of github.com:salesforce/LAVIS into main

    root committed May 25, 2023
    Configuration menu
    Copy the full SHA
    5f7585d View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2023

  1. finetune BLIP-2 for coco retrieval

    root committed Jul 3, 2023
    Configuration menu
    Copy the full SHA
    c0c6c47 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. update pnp vqa collab

    root committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    35923fe View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Update README.md

    @dxli94
    dxli94 authored Jul 12, 2023
    Configuration menu
    Copy the full SHA
    fceaaa2 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. 20230623 blip diffusion documentation (salesforce#441) 

    * BLIP-Diffusion
    @dxli94
    dxli94 authored Jul 21, 2023
    Configuration menu
    Copy the full SHA
    bbac655 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @dxli94
    dxli94 authored Jul 21, 2023
    Configuration menu
    Copy the full SHA
    2be8ee8 View commit details
    Browse the repository at this point in the history

  3. Add dependency installation to notebook (salesforce#442)

    @kopyl
    kopyl authored Jul 21, 2023
    Configuration menu
    Copy the full SHA
    7fe1dd5 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. [hotfix] salesforce#449 (salesforce#450)

    @dxli94
    dxli94 authored Jul 24, 2023
    Configuration menu
    Copy the full SHA
    61989d7 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. Update README.md (salesforce#452) 

    Updated outdated links in blip-diffusion's README.md
    @Jebiel
    Jebiel authored Jul 25, 2023
    Configuration menu
    Copy the full SHA
    54c2d64 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. blip diffusion editing and examples. (salesforce#462)

    @dxli94
    dxli94 authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    ea84b8b View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Update README.md

    @dxli94
    dxli94 authored Aug 18, 2023
    Configuration menu
    Copy the full SHA
    e6b5dd4 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. Update README.md

    @dxli94
    dxli94 authored Sep 25, 2023
    Configuration menu
    Copy the full SHA
    b693170 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Init Commit

    @artemisp
    artemisp committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    174ac24 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Update main LAVIS readme

    @artemisp
    artemisp committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    c7ad620 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @artemisp
    artemisp authored Dec 1, 2023
    Configuration menu
    Copy the full SHA
    50ea400 View commit details
    Browse the repository at this point in the history

  3. Update vicuna7b_v2.yaml

    @artemisp
    artemisp authored Dec 1, 2023
    Configuration menu
    Copy the full SHA
    f534093 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2023

  1. Update readme

    @artemisp
    artemisp committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    bb1a73b View commit details
    Browse the repository at this point in the history

  2. Update readme

    @artemisp
    artemisp committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    0c411c6 View commit details
    Browse the repository at this point in the history

  3. Add init files for processors,models,datasets were missed before

    @artemisp
    artemisp committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    0de4834 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Add missing files from earlier commit: mostly datasets, and updated t… 

    …ypos/cleaned up some comments
    @artemisp
    artemisp committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    adf49eb View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Updated README

    @artemisp
    artemisp committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    dee6dda View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Merge pull request salesforce#599 from artemisp/main 

    X-InstructBLIP Code
    @henryhungle
    henryhungle authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    b28106d View commit details
    Browse the repository at this point in the history

Commits on May 19, 2024

  1. update the requirements and fix the _expand_mask import issue

    @maulikmadhavi
    maulikmadhavi committed May 19, 2024
    Configuration menu
    Copy the full SHA
    f0317e0 View commit details
    Browse the repository at this point in the history

  2. update the requirements and fix the _expand_mask import issue

    Maulik Madhavi committed May 19, 2024
    Configuration menu
    Copy the full SHA
    ef2bf01 View commit details
    Browse the repository at this point in the history

  3. requirements update

    @maulikmadhavi
    maulikmadhavi committed May 19, 2024
    Configuration menu
    Copy the full SHA
    0871b23 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. Merge branch 'patch-requirements' of https://github.com/maulikmadhavi… 

    …/LAVIS into patch-requirements
    @maulikmadhavi
    maulikmadhavi committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    6a00768 View commit details
    Browse the repository at this point in the history