Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add simple benchmarking #74

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add simple benchmarking #74

wants to merge 3 commits into from

Commits on Dec 22, 2015

  1. Add simple benchmarking 

    Starting a function name with 'bench_' will now also store it and make
clar run it, but only when invoked with the '-b' flag.

In order to reduce how much clar overhead we measure, we put the timing
code around the function call, which means we end up timing every
function, but we only store the elapsed time in benchmark mode. The
overhead from calling the timer is minimal, so it shouldn't present an
issue to add these calls for every test.

This is not meant for micro-benchmarks, but for situations where we
expect the operations to take at least tens or hundreds of milliseconds
and we expect the operations not to be easily mapped into the "repeat N
times" method to suss out the noise, so we don't include this
functionality. This is for "perform very expensive operation" to track
high-scale changes though time.

Thus we also provide cl_reset_timer() to allow the function under test
to perform specific setup steps that it does not want to count towards
the timing.
    @carlosmn
    carlosmn committed Dec 22, 2015
    Configuration menu
    Copy the full SHA
    4a02a7f View commit details
    Browse the repository at this point in the history

  2. Report timings for all tests 

    Instead of separating the benchmark tests from "normal" tests, store
timings for all successful tests and let the user decide which tests
they'd like to run via the -s option.
    @carlosmn
    carlosmn committed Dec 22, 2015
    Configuration menu
    Copy the full SHA
    5e925f4 View commit details
    Browse the repository at this point in the history

  3. Show timing results at the end 

    Instead of intermixing benchmark results with the suite outputs, keep
them all until the end.
    @carlosmn
    carlosmn committed Dec 22, 2015
    Configuration menu
    Copy the full SHA
    7ea7444 View commit details
    Browse the repository at this point in the history