Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merging the new op: nlp_kv_cache_unpad_to_sharded into main #8459

Merged
merged 9 commits into from
May 16, 2024
Merged

Merging the new op: nlp_kv_cache_unpad_to_sharded into main #8459

merged 9 commits into from
May 16, 2024

Commits on May 16, 2024

  1. #7379: Added new nlp tm, unpad to sharded for kv cache

    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    0626976 View commit details
    Browse the repository at this point in the history

  2. #7379: added support for arbitrary number of batch-heads; resolved ha… 

    …ng on long seqlen; removed debug printout; added bfp8 test cases
    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    76e80cf View commit details
    Browse the repository at this point in the history

  3. #7379: optimized kernel to use less host side arguements and removed … 

    …unecessary variables and conditions in host code
    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    e6eac31 View commit details
    Browse the repository at this point in the history

  4. #7379: clean up host side code and moved static runtime args into com… 

    …pile time args
    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    6276f21 View commit details
    Browse the repository at this point in the history

  5. #7379: improved kernel perf with num core dependent async barriers; c… 

    …leaned up host side code
    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    653a499 View commit details
    Browse the repository at this point in the history

  6. #7379: resolved formatting issues

    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    e69e825 View commit details
    Browse the repository at this point in the history

  7. #7379: renamed op to nlp_kv_cache_load_slice

    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    480cbfc View commit details
    Browse the repository at this point in the history

  8. #7379: changed copy right to 2024

    @caixunshiren
    caixunshiren committed May 16, 2024
    Configuration menu
    Copy the full SHA
    43930e6 View commit details
    Browse the repository at this point in the history

  9. #7379: Added proper testing method for program caching

    @caixunshiren
    caixunshiren committed May 16, 2024
    1 Configuration menu
    Copy the full SHA
    17ae8f4 View commit details
    Browse the repository at this point in the history