Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modules/zstd: Add Repacketizer #1314

Closed
wants to merge 34 commits into from
Closed

Commits on Jan 15, 2024

  1. modules/zstd: Add buffer library

    This commit adds a DSLX Buffer library that provides the Buffer struct,
    and helper functions that can be used to operate on it. The Buffer
    is meant to be a storage for data coming from the channel. It acts like
    a FIFO, allowing data of any length to be put in or popped out of it.
    Provided DSLX tests verify the correct behaviour of the library.
    
    Internal-tag: [#50221]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    fef83f7 View commit details
    Browse the repository at this point in the history
  2. modules/zstd: Add Buffer use-case example

    This commit adds a simple test that shows, how one can use the Buffer
    struct inside a Proc.
    
    Internal-tag: [#50221]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    01f2dcd View commit details
    Browse the repository at this point in the history
  3. modules/zstd: Add library for parsing magic number

    This commit adds the library with functions for parsing a magic number and
    tests that verify its correctness.
    
    Internal-tag: [#50221]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    614f650 View commit details
    Browse the repository at this point in the history
  4. modules/zstd: Add library for parsing frame header

    This commit adds the library with functions for parsing a frame header.
    The provided tests verify the correcness of the library.
    
    Internal-tag: [#49967]
    Co-authored-by: Roman Dobrodii <rdobrodii@antmicro.com>
    Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    3 people committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    9838e81 View commit details
    Browse the repository at this point in the history
  5. modules/zstd/frame_header: Add benchmarking rules

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    09a7eba View commit details
    Browse the repository at this point in the history
  6. dependency_support/libzstd: Make zstd_errors.h public

    Required for expected_status inference in C++ tests for ZSTD decoder
    components
    
    Internal-tag: [#53465]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    4fbe593 View commit details
    Browse the repository at this point in the history
  7. dependency_support: Add decodecorpus binary

    Internal-tag: [#50967]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    f85661c View commit details
    Browse the repository at this point in the history
  8. modules/zstd: Add data generator library

    This commit adds a binary that calls decoding to generate data and loads
    it into a vector of bytes.
    
    Internal-tag: [#50967]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    5941efe View commit details
    Browse the repository at this point in the history
  9. modules/zstd: Add zstd frame header tests

    Internal-tag: [#50967]
    Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    rw1nkler and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    070089f View commit details
    Browse the repository at this point in the history
  10. modules/zstd: Add common zstd definitions

    Internal-tag: [#51343]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    05e4bf7 View commit details
    Browse the repository at this point in the history
  11. modules/zstd: Add raw block decoder

    Internal-tag: [#51343]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    80a9841 View commit details
    Browse the repository at this point in the history
  12. modules/zstd/raw_block_decoder: Add benchmarking rules

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    98cd055 View commit details
    Browse the repository at this point in the history
  13. modules/zstd: Add rle block decoder

    Adds RleBlockDecoder responsible for decoding Blocks
    of RLE_Block Block_Type as specified in RFC 8878, paragraph 3.1.1.2.2.
    https://datatracker.ietf.org/doc/html/rfc8878#section-3.1.1.2.2
    
    RleBlockDecoder communicates through BlockDataPacket channels.
    It reuses existing RunLengthDecoder block which is interfaced through
    two seprate procs:
    
     * RleDataPacker
     * BatchPacker
    
    Which are responsible for converting input data into format accepted by
    RLE decoder and for gathering RLE decoder output symbols into batches
    which are then send out through BlockDataPacket.
    
    Internal-tag: [#51473]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    4169107 View commit details
    Browse the repository at this point in the history
  14. modules/zstd/rle_block_dec: Specify fifo depths for internal channels

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    c72e115 View commit details
    Browse the repository at this point in the history
  15. modules/zstd/rle_block_decoder: Add benchmarking rules

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    80b5bc7 View commit details
    Browse the repository at this point in the history
  16. modules/zstd: Add block header parsing library

    Internal-tag: [#51343]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    5ef1e1d View commit details
    Browse the repository at this point in the history
  17. modules/zstd: Add SequenceExecutorPacket to common definitions

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    a89ca51 View commit details
    Browse the repository at this point in the history
  18. modules/zstd: Add block data muxer library

    This commit adds DecoderMux Proc, which collects data from specialized
    Raw, RLE, and Compressed Block decoders and re-sends them in the correct order.
    
    Internal-tag: [#51343]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    3d15529 View commit details
    Browse the repository at this point in the history
  19. modules/zstd/dec_mux: Add benchmarking rules

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    4552b37 View commit details
    Browse the repository at this point in the history
  20. modules/zstd: Add block demuxer library

    This DSLX proc responsibility is to dispatch encoded blocks to a correct
    decoder: RAW, RLE, COMPRESSED.
    
    It tracks and assigns block IDs.
    The ID counter is reset on the frame's last block on the last data packet.
    
    Internal-tag: [#51736]
    Co-authored-by: Robert Winkler <rwinkler@antmicro.com>
    Signed-off-by: Maciej Dudek <mdudek@antmicro.com>
    2 people authored and lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    327b2c1 View commit details
    Browse the repository at this point in the history
  21. modules/zstd/dec_demux: Specify fifo depths for internal channels

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    a9d9717 View commit details
    Browse the repository at this point in the history
  22. modules/zstd/dec_demux: Add benchmarking rules

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    34c711d View commit details
    Browse the repository at this point in the history
  23. modules/zstd: Add block decoder module

    This adds a decoder of block data. It decodes block header and
    demuxes remaining input data into one of specific block decoders
    depending on the type of the parsed block. Then it muxes outputs
    from those decoders into single output channel.
    
    Internal-tag: [#51873]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    43f3e6b View commit details
    Browse the repository at this point in the history
  24. modules/zstd/block_dec: Specify fifo depths for internal channels

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    8ee1391 View commit details
    Browse the repository at this point in the history
  25. modules/zstd/block_dec: Add benchmarking rules

    Internal-tag: [#53329]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    3b1da80 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. xls/moduels/common: Specify decoder output format

    Internal-tag: [#52954]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz authored and rw1nkler committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    86c5d9a View commit details
    Browse the repository at this point in the history
  2. examples/ram: Export internal RAM API to other modules

    This commit marks SimultaneousReadWriteBehavior enum and
    num_partitions function as public to allow for creating
    simpler tests that interact with RAM models.
    
    Internal-tag: [#53241]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    24f1ee2 View commit details
    Browse the repository at this point in the history
  3. modules/zstd: Add Offset type to common zstd definitions

    Internal-tag: [#54705]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    2486fae View commit details
    Browse the repository at this point in the history
  4. modules/zstd: Add RamPrinter Proc

    This commit adds RAM printer block usefull for debugging
    HistoryBuffer inside SequenceExecutor.
    
    Internal-tag: [#54705]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    0fc2f48 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. modules/zstd: Add SequenceExecutor Proc

    Add Proc responsible for handling ZSTD Sequence Execution step,
    which is described in:
    https://datatracker.ietf.org/doc/html/rfc8878#name-sequence-execution
    
    Internal-tag: [#54705]
    Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    rw1nkler committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    bd07cc1 View commit details
    Browse the repository at this point in the history
  2. modules/zstd: Add verilog generation and benchmarking rules for Seque…

    …nceExecutor
    
    Internal-tag: [#54705]
    rw1nkler committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    a0bffc4 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. xls/modules/zstd: Add repacketizer

    Internal-tag: [#52954]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    1c8035f View commit details
    Browse the repository at this point in the history
  2. xls/modules/zstd/BUILD: Add repacketizer build rules

    Internal-tag: [#52954]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    122edc5 View commit details
    Browse the repository at this point in the history
  3. xls/modules/zstd/BUILD: benchmark repacketizer

    Internal-tag: [#52954]
    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    lpawelcz committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    ba47228 View commit details
    Browse the repository at this point in the history