Skip to content

Commit

Permalink
quick sched yield test
Browse files Browse the repository at this point in the history
  • Loading branch information
LostRuins committed Jan 10, 2024
1 parent d2bf479 commit 4b59b87
Showing 1 changed file with 7 additions and 15 deletions.
22 changes: 7 additions & 15 deletions ggml.c
Original file line number Diff line number Diff line change
Expand Up @@ -16559,22 +16559,14 @@ static thread_ret_t ggml_graph_compute_thread(void * data) {
} else {
// wait for other threads to finish
const int last = node_n;

const bool do_yield = last < 0 || cgraph->nodes[last]->op == GGML_OP_MUL_MAT;

while (true) {
// TODO: this sched_yield can have significant impact on the performance - either positive or negative
// depending on the workload and the operating system.
// since it is not clear what is the best approach, it should potentially become user-configurable
// ref: https://github.com/ggerganov/ggml/issues/291
// UPD: adding the do_yield flag seems to resolve the issue universally
if (do_yield) {
sched_yield();
}

do {
#if defined(GGML_USE_ACCELERATE) || defined(GGML_USE_METAL)
//apple does nothing
#else
sched_yield();
#endif
node_n = atomic_load(&state->shared->node_n);
if (node_n != last) break;
};
} while (node_n == last);
}

// check if we should stop
Expand Down

0 comments on commit 4b59b87

Please sign in to comment.