Skip to content

Commit

Permalink
linux-gen: ring: fix ring size detection
Browse files Browse the repository at this point in the history
Previously, _RING_MPMC_ENQ_MULTI() and _RING_MPMC_ENQ_BATCH() could return
zero even though the ring was not actually full. Read w_head before r_tail,
which guarantees that w_head - r_tail <= size. Any additional delay in
reading r_tail makes the subtraction result only smaller. This avoids
returning zero when the ring is not actually full.

Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
Reported-by: Fan Hong <fan.hong@arm.com>
  • Loading branch information
MatiasElo committed Aug 24, 2023
1 parent 3acb203 commit ee1e983
Showing 1 changed file with 18 additions and 10 deletions.
28 changes: 18 additions & 10 deletions platform/linux-generic/include/odp_ring_mpmc_internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -212,13 +212,17 @@ static inline uint32_t _RING_MPMC_ENQ_MULTI(_ring_mpmc_gen_t *ring,
uint32_t old_head, new_head, r_tail, num_free, i;
uint32_t size = ring_mask + 1;

/* Load acquires ensure that w_head load happens after r_tail load,
* and thus r_tail value is always behind or equal to w_head value.
* When CAS operation succeeds, this thread owns data between old
* and new w_head. */
/* The CAS operation guarantees that w_head value is up to date. Load
* acquire is used to ensure that r_tail is read after w_head. This
* guarantees that w_head - r_tail <= size. Any additional delay in
* reading r_tail makes the subtraction result only smaller. This
* avoids returning zero when the ring is not actually full.
*
* When CAS operation succeeds, this thread owns data between old and
* new w_head. */
do {
r_tail = odp_atomic_load_acq_u32(&ring->r.r_tail);
old_head = odp_atomic_load_acq_u32(&ring->r.w_head);
r_tail = odp_atomic_load_acq_u32(&ring->r.r_tail);

num_free = size - (old_head - r_tail);

Expand Down Expand Up @@ -259,13 +263,17 @@ static inline uint32_t _RING_MPMC_ENQ_BATCH(_ring_mpmc_gen_t *ring,
uint32_t old_head, new_head, r_tail, num_free, i;
uint32_t size = ring_mask + 1;

/* Load acquires ensure that w_head load happens after r_tail load,
* and thus r_tail value is always behind or equal to w_head value.
* When CAS operation succeeds, this thread owns data between old
* and new w_head. */
/* The CAS operation guarantees that w_head value is up to date. Load
* acquire is used to ensure that r_tail is read after w_head. This
* guarantees that w_head - r_tail <= size. Any additional delay in
* reading r_tail makes the subtraction result only smaller. This
* avoids returning zero when the ring is not actually full.
*
* When CAS operation succeeds, this thread owns data between old and
* new w_head. */
do {
r_tail = odp_atomic_load_acq_u32(&ring->r.r_tail);
old_head = odp_atomic_load_acq_u32(&ring->r.w_head);
r_tail = odp_atomic_load_acq_u32(&ring->r.r_tail);

num_free = size - (old_head - r_tail);

Expand Down

0 comments on commit ee1e983

Please sign in to comment.