Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorRT parameter passing can be enhanced #366

Open
LeiWang1999 opened this issue Jun 26, 2022 · 0 comments
Open

TensorRT parameter passing can be enhanced #366

LeiWang1999 opened this issue Jun 26, 2022 · 0 comments

Comments

@LeiWang1999
Copy link

trtexec has a lot of arguments, but superbench only covers a fraction of them, and the default value trtexec set is not suitable for benchmarking our program, for example, when I use superbench to profile a resnet50.onnx with tensorrt backend, the command that superbench generated is :

/opt/tensorrt/bin/trtexec --onnx=/workspace/v-leiwang3/.torch/hub/onnx/resnet50.onnx --explicitBatch --optShapes=input:1x3x224x224 --workspace=8192 --iterations=105 --percentile=99.

How ever I found this command executed more than 200 executions on our V100 GPU, it was caused by the default arguments --duration was set to 3, which means trtexec will profile the model at least 3s, but for 100 iterations on resnet50, it only takes about 1.5 second, so the default value of --duration should be set to 0 to srtictly execute with given iterations.

And, for warmup step, trtexec also provides --warmUp options to set warmup step, so my expected command should be :

/opt/tensorrt/bin/trtexec --onnx=/workspace/v-leiwang3/.torch/hub/onnx/resnet50.onnx --explicitBatch --optShapes=input:1x3x224x224 --workspace=8192 --fp16 --avgRuns=10 --warmUp=5 --iterations=100 --percentile=99. --duration=0 
@cp5555 cp5555 mentioned this issue Aug 4, 2022
27 tasks
@cp5555 cp5555 mentioned this issue Oct 24, 2022
20 tasks
@cp5555 cp5555 mentioned this issue Jan 28, 2023
21 tasks
@cp5555 cp5555 mentioned this issue Jul 26, 2023
30 tasks
@cp5555 cp5555 mentioned this issue Mar 27, 2024
22 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant