[BUG] BenchConfig arguments in Bench seem to have no effect

[samufi]

Bug description

When benchmarking a simple function, the constraints set as arguments of BenchConfig, handed over to Bench seem not to have an effect. On my machine, the benchmarking does not halt in reasonable time. (I have not observed the program to stop at all, but I have not waited longer than 30s)

Steps to reproduce

Reproduce via the following code:

import random
from benchmark import Bencher, keep, Bench, BenchId, BenchConfig


fn expensive_fn() -> Int:
    var s = 0
    for _ in range(100):
        s += int(random.random_float64() < 0.5)
    return s


fn benchmark_fun(inout bencher: Bencher) capturing:
    @always_inline
    @parameter
    fn bench_fn(calls: Int) capturing -> Int:
        for _ in range(calls):
            keep(expensive_fn()) 
        return calls

    bencher.iter_custom[bench_fn]()
 
def main():
    config = BenchConfig(max_runtime_secs=3, max_iters=3)
    bench = Bench(config)
    bench.bench_function[benchmark_fun](BenchId("my_benchmark"))
    bench.dump_report()

Also, when adding print statements to bench_fn, it can be seen that bench_fn is never called with an argument other than 1, which I find unexpected.

System information

- Ubuntu 22.04.5 LTS in WSL2 on Windows 10
- mojo 2024.11.1705 (43b6e3df)
- magic 0.4.0 - (based on pixi 0.33.0)

[samufi]

Because I could not find the source code for the benchmarking module (please let me know if I overlooked something), I made my own small implementation. Let me know if this could be of any help for comparing the actual implementation with one that works =).