libtorch jit forward error

[kli017]

❓ Questions and Help

Hello I was implement the new vad model with libtorch jit. The init and forward code as bellow
`
static float ComputeConfidence(const std::vectortorch::jit::IValue &inputs, const int sr) {
torch::NoGradGuard no_grad;
auto output = model->forward(inputs).toTensor();
return output[0][1].item().toFloat();
}

bool InitTorchVadModel(const TorchVadOptions &opts) {
model = std::make_shared(
std::move(torch::jit::load(opts.model_path)));
model->eval();
model->to(at::kCPU);
at::set_num_threads(opts.num_thread);
return model != nullptr;
}
`
while make the project I got the error as below, which said that the forward function need a parameter named sr.
terminate called after throwing an instance of 'c10::Error'
what(): forward() is missing value for argument 'sr'. Declaration: forward(torch.HehRecModel self, Tensor x, int sr) -> (Tensor)
Exception raised from checkAndNormalizeInputs at ../aten/src/ATen/core/function_schema_inl.h:239 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x69 (0x7f2e48c5d9f9 in /root/kli/libtorch_vad/fc_base/libtorch-src/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0xd2 (0x7f2e48c5a012 in /root/kli/libtorch_vad/fc_base/libtorch-src/lib/libc10.so)
frame #2: + 0xf10708 (0x7f2e368db708 in /root/kli/libtorch_vad/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #3: torch::jit::GraphFunction::operator()(std::vector<c10::IValue, std::allocatorc10::IValue >, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, c10::IValue, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, c10::IValue> > > const&) + 0x2d (0x7f2e39209b2d in /root/kli/libtorch_vad/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #4: torch::jit::Method::operator()(std::vector<c10::IValue, std::allocatorc10::IValue >, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, c10::IValue, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, c10::IValue> > > const&) + 0x16a (0x7f2e39217f8a in /root/kli/libtorch_vad/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #5: torch::jit::Module::forward(std::vector<c10::IValue, std::allocatorc10::IValue >) + 0xff (0x7f2e48874e3b in liblibtorch_vad.so)
frame #6: + 0x3184d (0x7f2e4886f84d in liblibtorch_vad.so)
frame #7: voicecomm::vad::TorchVad::IsSpeech(std::vector<float, std::allocator > const&) + 0x13f (0x7f2e4886fd4d in liblibtorch_vad.so)
frame #8: voicecomm::vad::TorchVad::IsSpeech(std::vector<char, std::allocator > const&) + 0x46 (0x7f2e4886fbc6 in liblibtorch_vad.so)
frame #9: + 0x2b47 (0x5577003e9b47 in ./libtorch_vad_main)
frame #10: __libc_start_main + 0xe7 (0x7f2e3505abf7 in /lib/x86_64-linux-gnu/libc.so.6)
frame #11: + 0x286a (0x5577003e986a in ./libtorch_vad_main)

[snakers4]

while make the project I got the error as below, which said that the forward function need a parameter named sr.

Yes, this is correct

silero-vad/utils_vad.py

Lines 205 to 211 in 76687cb

	speech_probs = []
	for current_start_sample in range(0, audio_length_samples, window_size_samples):
	chunk = audio[current_start_sample: current_start_sample + window_size_samples]
	if len(chunk) < window_size_samples:
	chunk = torch.nn.functional.pad(chunk, (0, int(window_size_samples - len(chunk))))
	speech_prob = model(chunk, sampling_rate).item()
	speech_probs.append(speech_prob)

[snakers4]

error: no matching function for call to ‘torch::jit::Module::forward(const std::vectorc10::IValue&, const int&)
note: candidate expects 1 argument, 2 provided

To be honest this is a bit cryptic to me, looks like to me that it complains about actually having 2 parameters instead of one
Maybe there some limitations in libtorch or model.forward has to be invoked differently if it has several params?

[kli017]

Yes, The previous jit model which only need inputdata works well. no idea how to deal with the error.

[snakers4]

Maybe this would help - pytorch/pytorch#18337 (comment)

[kli017]

Thanks! So should I convert the sr from int to a tensor and put it to the inputs? I tried so but got what(): forward() Expected a value of type 'int' for argument 'sr' but instead found type 'Tensor'.
Seems the int sr should be an independet paramter..

[snakers4]

They mention creating a special IValue object
I am not familiar with C++ API, but looks like a dictionary to me