Skip to content
This repository has been archived by the owner on Aug 30, 2024. It is now read-only.

[SYCL] Support SYCL layer for LLaMA2 model #272

Merged
merged 77 commits into from
Jun 21, 2024
Merged

[SYCL] Support SYCL layer for LLaMA2 model #272

merged 77 commits into from
Jun 21, 2024
+3,733 −1,266

Commits on Jun 21, 2024

  1. fixed all UTs

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    23aad22 View commit details
    Browse the repository at this point in the history

  2. move sycl benchmark to benchmark project

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    83592d1 View commit details
    Browse the repository at this point in the history

  3. add q4 UT for sycl prologue_b

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1d6914e View commit details
    Browse the repository at this point in the history

  4. sycl gemv case

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c29095a View commit details
    Browse the repository at this point in the history

  5. add UT case

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    365697c View commit details
    Browse the repository at this point in the history

  6. add to trans storage conversion

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0c1b0fe View commit details
    Browse the repository at this point in the history

  7. add sycl context to model context. compile ne_layers with dpcpp

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    5ede223 View commit details
    Browse the repository at this point in the history

  8. add context sycl memory allocation

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    ac79c6c View commit details
    Browse the repository at this point in the history

  9. inference with data exchange

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d533fc8 View commit details
    Browse the repository at this point in the history

  10. use backend instead of ne type

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    804f4fc View commit details
    Browse the repository at this point in the history

  11. new api to assign sycl buffer

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    9dfd12a View commit details
    Browse the repository at this point in the history

  12. add backend parameter for new tensor

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    cf3e0ee View commit details
    Browse the repository at this point in the history

  13. add sycl int4 to graph compute

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a20a5d3 View commit details
    Browse the repository at this point in the history

  14. fix sync

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1666286 View commit details
    Browse the repository at this point in the history

  15. compile without sycl

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    09c97ee View commit details
    Browse the repository at this point in the history

  16. sync with main

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    f835c11 View commit details
    Browse the repository at this point in the history

  17. fix tensor size bug

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    793cbea View commit details
    Browse the repository at this point in the history

  18. refactor layer config

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d685cfa View commit details
    Browse the repository at this point in the history

  19. sync ut

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    db905fd View commit details
    Browse the repository at this point in the history

  20. support 2 layers of sycl

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a5c9c7f View commit details
    Browse the repository at this point in the history

  21. sync main

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c400b9c View commit details
    Browse the repository at this point in the history

  22. revert ISA detect for dpcpp

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    b0fe7ea View commit details
    Browse the repository at this point in the history

  23. compile without dpcpp

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c0dcbdf View commit details
    Browse the repository at this point in the history

  24. fix avxvnni intrin code

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    40e6e21 View commit details
    Browse the repository at this point in the history

  25. protect crash if it's a CPU SYCL device

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1789f3f View commit details
    Browse the repository at this point in the history

  26. add device mul function

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    7e64369 View commit details
    Browse the repository at this point in the history

  27. fix the sync issue

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0ae743e View commit details
    Browse the repository at this point in the history

  28. run model with all FFN layers on SYCL

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4e26650 View commit details
    Browse the repository at this point in the history

  29. fix compile

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    479bc87 View commit details
    Browse the repository at this point in the history

  30. clang-format

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    da3c676 View commit details
    Browse the repository at this point in the history

  31. revert model config

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c9b46ed View commit details
    Browse the repository at this point in the history

  32. fix fun ret

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    da787cb View commit details
    Browse the repository at this point in the history

  33. fix the kernel bug

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    b433333 View commit details
    Browse the repository at this point in the history

  34. remove all grad tensors.

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    01737dc View commit details
    Browse the repository at this point in the history

  35. fix some bugs.

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    ecc1cbe View commit details
    Browse the repository at this point in the history

  36. support llama shapes, add new UT case, update new api of dpcpp

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a7aab9c View commit details
    Browse the repository at this point in the history

  37. support all ffn layers

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    cd60d31 View commit details
    Browse the repository at this point in the history

  38. add sync for CPU Device

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    45e56f5 View commit details
    Browse the repository at this point in the history

  39. clang-format

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0ed2228 View commit details
    Browse the repository at this point in the history

  40. fix warning

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    f73ee66 View commit details
    Browse the repository at this point in the history

  41. clang-format

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c43f2ef View commit details
    Browse the repository at this point in the history

  42. add back f32 model support

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    ef9e365 View commit details
    Browse the repository at this point in the history

  43. fix typo, remove unused code

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    99c41c0 View commit details
    Browse the repository at this point in the history

  44. bring more layers to SYCL

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c30c23f View commit details
    Browse the repository at this point in the history

  45. add embedding support and use omp in sycl

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    8890f50 View commit details
    Browse the repository at this point in the history

  46. optimize gemv k iteration

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a189d90 View commit details
    Browse the repository at this point in the history

  47. optimize rms_norm, add debug macro for no-mha forward.

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    b9e7aa2 View commit details
    Browse the repository at this point in the history

  48. add mha ut

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d908ac8 View commit details
    Browse the repository at this point in the history

  49. prepare for SYCL MHA

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    fd1fbef View commit details
    Browse the repository at this point in the history

  50. add SYCL rope

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    52285a9 View commit details
    Browse the repository at this point in the history

  51. all device f32 mha

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    03da9a8 View commit details
    Browse the repository at this point in the history

  52. remove unused code

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    89da98b View commit details
    Browse the repository at this point in the history

  53. fixed

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0323edd View commit details
    Browse the repository at this point in the history

  54. fixed

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    360f099 View commit details
    Browse the repository at this point in the history

  55. refactor sycl context for multiple allocation

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    ebfa1d5 View commit details
    Browse the repository at this point in the history

  56. support n_gpu_layer

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    fb8c397 View commit details
    Browse the repository at this point in the history

  57. reuse scratch

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    13a8813 View commit details
    Browse the repository at this point in the history

  58. add new mha version

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    ab3d76c View commit details
    Browse the repository at this point in the history

  59. new version of MHA

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4d254eb View commit details
    Browse the repository at this point in the history

  60. lower malloc size

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    023d78c View commit details
    Browse the repository at this point in the history

  61. compile without sycl

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    72f71c5 View commit details
    Browse the repository at this point in the history

  62. run llama without sycl build

    @ThanatosShinji @luoyu-intel
    ThanatosShinji authored and luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    e881934 View commit details
    Browse the repository at this point in the history

  63. clang-format

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    243d744 View commit details
    Browse the repository at this point in the history

  64. fix clang-tidy

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    e4e64e0 View commit details
    Browse the repository at this point in the history

  65. fix py build

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    2aae1ba View commit details
    Browse the repository at this point in the history

  66. fix warning

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    3d36da0 View commit details
    Browse the repository at this point in the history

  67. use std header

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    5028cf2 View commit details
    Browse the repository at this point in the history

  68. update math

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    f3e6eb4 View commit details
    Browse the repository at this point in the history

  69. update math

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    234b157 View commit details
    Browse the repository at this point in the history

  70. revert scratch without SYCL

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a580f56 View commit details
    Browse the repository at this point in the history

  71. use cl for c_compiler

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    116073a View commit details
    Browse the repository at this point in the history

  72. compile on linux

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    481dd4f View commit details
    Browse the repository at this point in the history

  73. Revert "compile on linux" 

    This reverts commit 0ce1574.
    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    f37e04d View commit details
    Browse the repository at this point in the history

  74. Revert "use cl for c_compiler" 

    This reverts commit 7f40ae9.
    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    145dc98 View commit details
    Browse the repository at this point in the history

  75. fix memory leak, set lower extra memory size.

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    42fa774 View commit details
    Browse the repository at this point in the history

  76. revert embedding size on CPU

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    5d02676 View commit details
    Browse the repository at this point in the history

  77. clang-format

    @luoyu-intel
    luoyu-intel committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    5bee840 View commit details
    Browse the repository at this point in the history