使用 generate-int8-scale-table 崩溃 （Segmentation fault）

[wanganddi0918]

请教一下，说明手册 https://k2-fsa.github.io/icefall/model-export/export-ncnn-conv-emformer.html#optional-int8-quantization-with-sherpa-ncnn 中是描述如何将 *bin 和 *param 文件转成对应的 int8 轻量化形式，但我遇到了一个问题：

在使用命令时，发生了崩溃（Segmentation fault）：
generate-int8-scale-table
./encoder_jit_trace-pnnx.ncnn.param
./encoder_jit_trace-pnnx.ncnn.bin
./decoder_jit_trace-pnnx.ncnn.param
./decoder_jit_trace-pnnx.ncnn.bin
./joiner_jit_trace-pnnx.ncnn.param
./joiner_jit_trace-pnnx.ncnn.bin
./encoder-scale-table.txt
./joiner-scale-table.txt
./wave_filenames.txt

于是我打印了一下脚本 generate-int8-scale-table.cc 中 convolution->weight_data.data 的值为 nullptr。请问这可能是什么原因导致的呢？

具体操作流程如下：

1.Download the pre-trained model

---

mkdir -p /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/data/lang_bpe_500
cd /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05
wget https://hf-mirror.com/Zengwei/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/resolve/main/exp/pretrained-epoch-30-avg-10-averaged.pt?download=true
mv pretrained-epoch-30-avg-10-averaged.pt?download=true pretrained-epoch-30-avg-10-averaged.pt
wget https://hf-mirror.com/Zengwei/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/resolve/main/data/lang_bpe_500/bpe.model?download=true
mv bpe.model?download=true data/lang_bpe_500/bpe.model
wget https://huggingface.co/Zengwei/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/resolve/main/data/lang_bpe_500/tokens.txt?download=true
mv tokens.txt?download=true data/lang_bpe_500/tokens.txt

2. Install ncnn and pnnx

---

cd /home/ncnn
git clone https://github.com/csukuangfj/ncnn
cd ncnn
git submodule update --recursive --init

mkdir -p build-wheel
cd build-wheel
cmake
-DCMAKE_BUILD_TYPE=Release
-DNCNN_PYTHON=ON
-DNCNN_BUILD_BENCHMARK=OFF
-DNCNN_BUILD_EXAMPLES=OFF
-DNCNN_BUILD_TOOLS=ON
..
make -j4
cd ..

export PYTHONPATH=$PWD/python:$PYTHONPATH
export PATH=$PWD/tools/pnnx/build/src:$PATH
export PATH=$PWD/build-wheel/tools/quantize:$PATH

cd tools/pnnx
mkdir build
cd build
cmake -DCMAKE_CUDA_COMPILER="/usr/local/cuda/bin/nvcc" ..
make -j4
./src/pnnx

3. Export the model via torch.jit.trace()

---

cd /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05
mkdir exp
ln -s pretrained-epoch-30-avg-10-averaged.pt ./exp/epoch-30.pt
cd /home/icefall/egs/librispeech/ASR

export PYTHONPATH=$PYTHONPATH:/home/icefall:/home/ncnn/ncnn/python/ncnn:/home/ncnn/ncnn/build-wheel/python/ncnn
dir=/home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05
./conv_emformer_transducer_stateless2/export-for-ncnn.py
--exp-dir $dir/exp
--tokens $dir/data/lang_bpe_500/tokens.txt
--epoch 30
--avg 1
--use-averaged-model 0
--num-encoder-layers 12
--chunk-length 32
--cnn-module-kernel 31
--left-context-length 32
--right-context-length 8
--memory-size 32
--encoder-dim 512

输出如下：
2024-11-19 11:08:12,605 INFO [export-for-ncnn.py:225] device: cpu
2024-11-19 11:08:12,606 INFO [export-for-ncnn.py:234] {
"avg": 1,
"batch_idx_train": 0,
"best_train_epoch": -1,
"best_train_loss": Infinity,
"best_valid_epoch": -1,
"best_valid_loss": Infinity,
"blank_id": 0,
"chunk_length": 32,
"cnn_module_kernel": 31,
"context_size": 2,
"decoder_dim": 512,
"dim_feedforward": 2048,
"encoder_dim": 512,
"env_info": {
"IP address": "10.10.20.22",
"hostname": "gpu22.unidev.ai",
"icefall-git-branch": "master",
"icefall-git-date": "Fri Nov 1 22:49:19 2024",
"icefall-git-sha1": "57451b03-dirty",
"icefall-path": "/home/icefall",
"k2-build-type": "Release",
"k2-git-date": "Fri Feb 23 01:48:38 2024",
"k2-git-sha1": "ff1d435a8d3c4eaa15828a84a7240678a70539a7",
"k2-path": "/home/wangdi/.conda/envs/wd_vad/lib/python3.10/site-packages/k2/init.py",
"k2-version": "1.24.4",
"k2-with-cuda": true,
"lhotse-path": "/home/wangdi/.conda/envs/wd_vad/lib/python3.10/site-packages/lhotse/init.py",
"lhotse-version": "1.27.0",
"python-version": "3.10",
"torch-cuda-available": true,
"torch-cuda-version": "11.7",
"torch-version": "2.0.1+cu117"
},
"epoch": 30,
"exp_dir": "/home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/exp",
"feature_dim": 80,
"is_pnnx": true,
"iter": 0,
"joiner_dim": 512,
"left_context_length": 32,
"log_interval": 50,
"memory_size": 32,
"model_warm_step": 3000,
"nhead": 8,
"num_encoder_layers": 12,
"reset_interval": 200,
"right_context_length": 8,
"subsampling_factor": 4,
"tokens": "/home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//data/lang_bpe_500/tokens.txt",
"use_averaged_model": false,
"valid_interval": 3000,
"vocab_size": 500
}
2024-11-19 11:08:12,607 INFO [export-for-ncnn.py:236] About to create model
2024-11-19 11:08:13,438 INFO [checkpoint.py:112] Loading checkpoint from /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/exp/epoch-30.pt
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:325] encoder parameters: 73937904
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:326] decoder parameters: 257024
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:327] joiner parameters: 781812
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:328] total parameters: 74976740
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:330] Using torch.jit.trace()
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:332] Exporting encoder
2024-11-19 11:08:14,104 INFO [export-for-ncnn.py:152] chunk_length: 32, right_context_length: 8

/home/icefall/egs/librispeech/ASR/conv_emformer_transducer_stateless2/emformer2.py:614: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert attention.shape == (B * self.nhead, Q, self.head_dim)
/home/icefall/egs/librispeech/ASR/conv_emformer_transducer_stateless2/emformer2.py:405: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert cache.shape == (B, D, self.cache_size), cache.shape
/home/wangdi/.conda/envs/wd_vad/lib/python3.10/site-packages/torch/jit/_trace.py:1056: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for list, use a tuple instead. for dict, use a NamedTuple instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
module._c._create_method_from_trace(
2024-11-19 11:08:17,300 INFO [export-for-ncnn.py:161] Saved to /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/exp/encoder_jit_trace-pnnx.pt
2024-11-19 11:08:17,302 INFO [export-for-ncnn.py:336] Exporting decoder
/home/icefall/egs/librispeech/ASR/conv_emformer_transducer_stateless2/decoder.py:100: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
need_pad = bool(need_pad)
2024-11-19 11:08:17,317 INFO [export-for-ncnn.py:183] Saved to /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/exp/decoder_jit_trace-pnnx.pt
2024-11-19 11:08:17,318 INFO [export-for-ncnn.py:340] Exporting joiner
2024-11-19 11:08:17,332 INFO [export-for-ncnn.py:210] Saved to /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/exp/joiner_jit_trace-pnnx.pt

4. Export torchscript model via pnnx

---

/home/ncnn/ncnn/tools/pnnx/build/src/pnnx $dir/exp/decoder_jit_trace-pnnx.pt
/home/ncnn/ncnn/tools/pnnx/build/src/pnnx $dir/exp/encoder_jit_trace-pnnx.pt fp16=0
/home/ncnn/ncnn/tools/pnnx/build/src/pnnx $dir/exp/joiner_jit_trace-pnnx.pt fp16=0

输出如下：
pnnxparam = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx.pnnx.param
pnnxbin = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx.pnnx.bin
pnnxpy = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx_pnnx.py
pnnxonnx = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx.pnnx.onnx
ncnnparam = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx.ncnn.param
ncnnbin = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx.ncnn.bin
ncnnpy = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/decoder_jit_trace-pnnx_ncnn.py
fp16 = 1
optlevel = 2
device = cpu
inputshape =
inputshape2 =
customop =
moduleop = scaling_converter.PoolingModuleNoProj,zipformer.AttentionDownsampleUnsqueeze,zipformer_for_ncnn_export_only.AttentionDownsampleUnsqueeze
############# pass_level0
############# pass_level1
############# pass_level2
############# pass_level3
open failed
############# pass_level4
############# pass_level5
pnnx build without onnx-zero support, skip saving onnx
############# pass_ncnn
pnnxparam = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx.pnnx.param
pnnxbin = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx.pnnx.bin
pnnxpy = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx_pnnx.py
pnnxonnx = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx.pnnx.onnx
ncnnparam = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx.ncnn.param
ncnnbin = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx.ncnn.bin
ncnnpy = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/encoder_jit_trace-pnnx_ncnn.py
fp16 = 0
optlevel = 2
device = cpu
inputshape =
inputshape2 =
customop =
moduleop = scaling_converter.PoolingModuleNoProj,zipformer.AttentionDownsampleUnsqueeze,zipformer_for_ncnn_export_only.AttentionDownsampleUnsqueeze
############# pass_level0
inline module = emformer2.Conv2dSubsampling
inline module = scaling.DoubleSwish
inline module = scaling_converter.NonScaledNorm
inline module = torch.nn.modules.linear.Identity
############# pass_level1
############# pass_level2
############# pass_level3
open failed
############# pass_level4
############# pass_level5
make_slice_expression input 67
make_slice_expression input 885
make_slice_expression input 90
make_slice_expression input 887
make_slice_expression input 94
make_slice_expression input 889
make_slice_expression input 116
make_slice_expression input 884
make_slice_expression input 157
make_slice_expression input 893
make_slice_expression input 161
make_slice_expression input 895
make_slice_expression input 183
make_slice_expression input 883
make_slice_expression input 224
make_slice_expression input 899
make_slice_expression input 228
make_slice_expression input 901
make_slice_expression input 250
make_slice_expression input 882
make_slice_expression input 291
make_slice_expression input 905
make_slice_expression input 295
make_slice_expression input 907
make_slice_expression input 317
make_slice_expression input 881
make_slice_expression input 358
make_slice_expression input 911
make_slice_expression input 362
make_slice_expression input 913
make_slice_expression input 384
make_slice_expression input 880
make_slice_expression input 425
make_slice_expression input 917
make_slice_expression input 429
make_slice_expression input 919
make_slice_expression input 451
make_slice_expression input 879
make_slice_expression input 492
make_slice_expression input 923
make_slice_expression input 496
make_slice_expression input 925
make_slice_expression input 518
make_slice_expression input 878
make_slice_expression input 559
make_slice_expression input 929
make_slice_expression input 563
make_slice_expression input 931
make_slice_expression input 585
make_slice_expression input 877
make_slice_expression input 626
make_slice_expression input 935
make_slice_expression input 630
make_slice_expression input 937
make_slice_expression input 652
make_slice_expression input 876
make_slice_expression input 693
make_slice_expression input 941
make_slice_expression input 697
make_slice_expression input 943
make_slice_expression input 719
make_slice_expression input 875
make_slice_expression input 760
make_slice_expression input 947
make_slice_expression input 764
make_slice_expression input 949
make_slice_expression input 786
make_slice_expression input 874
make_slice_expression input 827
make_slice_expression input 953
make_slice_expression input 831
make_slice_expression input 955
make_slice_expression input 853
make_slice_expression input 873
pnnx build without onnx-zero support, skip saving onnx
############# pass_ncnn
fallback batch axis 233 for operand 87
fallback batch axis 233 for operand 96
fallback batch axis 233 for operand 91
fallback batch axis 233 for operand 98
fallback batch axis 233 for operand 95
fallback batch axis 233 for operand 102
fallback batch axis 233 for operand 99
fallback batch axis 233 for operand 100
fallback batch axis 233 for operand 101
fallback batch axis 233 for operand 103
fallback batch axis 233 for operand 104
fallback batch axis 233 for operand 154
fallback batch axis 233 for operand 163
fallback batch axis 233 for operand 158
fallback batch axis 233 for operand 165
fallback batch axis 233 for operand 162
fallback batch axis 233 for operand 169
fallback batch axis 233 for operand 166
fallback batch axis 233 for operand 167
fallback batch axis 233 for operand 168
fallback batch axis 233 for operand 170
fallback batch axis 233 for operand 171
fallback batch axis 233 for operand 221
fallback batch axis 233 for operand 230
fallback batch axis 233 for operand 225
fallback batch axis 233 for operand 232
fallback batch axis 233 for operand 229
fallback batch axis 233 for operand 236
fallback batch axis 233 for operand 233
fallback batch axis 233 for operand 234
fallback batch axis 233 for operand 235
fallback batch axis 233 for operand 237
fallback batch axis 233 for operand 238
fallback batch axis 233 for operand 288
fallback batch axis 233 for operand 297
fallback batch axis 233 for operand 292
fallback batch axis 233 for operand 299
fallback batch axis 233 for operand 296
fallback batch axis 233 for operand 303
fallback batch axis 233 for operand 300
fallback batch axis 233 for operand 301
fallback batch axis 233 for operand 302
fallback batch axis 233 for operand 304
fallback batch axis 233 for operand 305
fallback batch axis 233 for operand 355
fallback batch axis 233 for operand 364
fallback batch axis 233 for operand 359
fallback batch axis 233 for operand 366
fallback batch axis 233 for operand 363
fallback batch axis 233 for operand 370
fallback batch axis 233 for operand 367
fallback batch axis 233 for operand 368
fallback batch axis 233 for operand 369
fallback batch axis 233 for operand 371
fallback batch axis 233 for operand 372
fallback batch axis 233 for operand 422
fallback batch axis 233 for operand 431
fallback batch axis 233 for operand 426
fallback batch axis 233 for operand 433
fallback batch axis 233 for operand 430
fallback batch axis 233 for operand 437
fallback batch axis 233 for operand 434
fallback batch axis 233 for operand 435
fallback batch axis 233 for operand 436
fallback batch axis 233 for operand 438
fallback batch axis 233 for operand 439
fallback batch axis 233 for operand 489
fallback batch axis 233 for operand 498
fallback batch axis 233 for operand 493
fallback batch axis 233 for operand 500
fallback batch axis 233 for operand 497
fallback batch axis 233 for operand 504
fallback batch axis 233 for operand 501
fallback batch axis 233 for operand 502
fallback batch axis 233 for operand 503
fallback batch axis 233 for operand 505
fallback batch axis 233 for operand 506
fallback batch axis 233 for operand 556
fallback batch axis 233 for operand 565
fallback batch axis 233 for operand 560
fallback batch axis 233 for operand 567
fallback batch axis 233 for operand 564
fallback batch axis 233 for operand 571
fallback batch axis 233 for operand 568
fallback batch axis 233 for operand 569
fallback batch axis 233 for operand 570
fallback batch axis 233 for operand 572
fallback batch axis 233 for operand 573
fallback batch axis 233 for operand 623
fallback batch axis 233 for operand 632
fallback batch axis 233 for operand 627
fallback batch axis 233 for operand 634
fallback batch axis 233 for operand 631
fallback batch axis 233 for operand 638
fallback batch axis 233 for operand 635
fallback batch axis 233 for operand 636
fallback batch axis 233 for operand 637
fallback batch axis 233 for operand 639
fallback batch axis 233 for operand 640
fallback batch axis 233 for operand 690
fallback batch axis 233 for operand 699
fallback batch axis 233 for operand 694
fallback batch axis 233 for operand 701
fallback batch axis 233 for operand 698
fallback batch axis 233 for operand 705
fallback batch axis 233 for operand 702
fallback batch axis 233 for operand 703
fallback batch axis 233 for operand 704
fallback batch axis 233 for operand 706
fallback batch axis 233 for operand 707
fallback batch axis 233 for operand 757
fallback batch axis 233 for operand 766
fallback batch axis 233 for operand 761
fallback batch axis 233 for operand 768
fallback batch axis 233 for operand 765
fallback batch axis 233 for operand 772
fallback batch axis 233 for operand 769
fallback batch axis 233 for operand 770
fallback batch axis 233 for operand 771
fallback batch axis 233 for operand 773
fallback batch axis 233 for operand 774
fallback batch axis 233 for operand 824
fallback batch axis 233 for operand 833
fallback batch axis 233 for operand 828
fallback batch axis 233 for operand 835
fallback batch axis 233 for operand 832
fallback batch axis 233 for operand 839
fallback batch axis 233 for operand 836
fallback batch axis 233 for operand 837
fallback batch axis 233 for operand 838
fallback batch axis 233 for operand 840
fallback batch axis 233 for operand 841
fallback batch axis 233 for operand pnnx_expr_2212_mul(96,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_2026_mul(163,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_1836_mul(230,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_1646_mul(297,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_1456_mul(364,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_1266_mul(431,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_1076_mul(498,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_886_mul(565,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_696_mul(632,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_506_mul(699,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_316_mul(766,1.250000e-01)
fallback batch axis 233 for operand pnnx_expr_126_mul(833,1.250000e-01)
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
reshape tensor with batch index 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
unsqueeze batch dim 1 is not supported yet!
pnnxparam = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx.pnnx.param
pnnxbin = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx.pnnx.bin
pnnxpy = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx_pnnx.py
pnnxonnx = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx.pnnx.onnx
ncnnparam = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx.ncnn.param
ncnnbin = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx.ncnn.bin
ncnnpy = /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05//exp/joiner_jit_trace-pnnx_ncnn.py
fp16 = 0
optlevel = 2
device = cpu
inputshape =
inputshape2 =
customop =
moduleop = scaling_converter.PoolingModuleNoProj,zipformer.AttentionDownsampleUnsqueeze,zipformer_for_ncnn_export_only.AttentionDownsampleUnsqueeze
############# pass_level0
############# pass_level1
############# pass_level2
############# pass_level3
open failed
############# pass_level4
############# pass_level5
pnnx build without onnx-zero support, skip saving onnx
############# pass_ncnn
fallback batch axis 233 for operand 0
fallback batch axis 233 for operand 1
fallback batch axis 233 for operand 2
fallback batch axis 233 for operand 3
fallback batch axis 233 for operand 5
fallback batch axis 233 for operand 6
fallback batch axis 233 for operand pnnx_expr_0_add(2,3)

5. Test the exported models in icefall

---

./conv_emformer_transducer_stateless2/streaming-ncnn-decode.py
--tokens $dir/data/lang_bpe_500/tokens.txt
--encoder-param-filename $dir/exp/encoder_jit_trace-pnnx.ncnn.param
--encoder-bin-filename $dir/exp/encoder_jit_trace-pnnx.ncnn.bin
--decoder-param-filename $dir/exp/decoder_jit_trace-pnnx.ncnn.param
--decoder-bin-filename $dir/exp/decoder_jit_trace-pnnx.ncnn.bin
--joiner-param-filename $dir/exp/joiner_jit_trace-pnnx.ncnn.param
--joiner-bin-filename $dir/exp/joiner_jit_trace-pnnx.ncnn.bin
$dir/test_wavs/1089-134686-0001.wav

输出：
2024-11-19 11:08:32,487 INFO [streaming-ncnn-decode.py:326] Constructing Fbank computer
2024-11-19 11:08:32,488 INFO [streaming-ncnn-decode.py:329] Reading sound files: /home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/test_wavs/1089-134686-0001.wav
2024-11-19 11:08:32,489 INFO [streaming-ncnn-decode.py:334] torch.Size([106000])
2024-11-19 11:08:34,085 INFO [streaming-ncnn-decode.py:378] /data3/unisound/wangdi/icefall/egs/aishell/mix_models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/test_wavs/1089-134686-0001.wav
2024-11-19 11:08:34,085 INFO [streaming-ncnn-decode.py:379] AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS

6. Modify the exported encoder for sherpa-ncnn

---

源文件：
7767517
1060 1341
Input in0 0 1 in0
Input in1 0 1 in1

修改为：
7767517
1061 1341
SherpaMetaData sherpa_meta_data1 0 0 0=1 1=12 2=32 3=31 4=8 5=32 6=8 7=512
Input in0 0 1 in0
Input in1 0 1 in1

cd /home/sherpa-ncnn
git clone https://github.com/k2-fsa/sherpa-ncnn
cd sherpa-ncnn
mkdir build
cd build
cmake ..
make -j 4

./bin/generate-int8-scale-table
export PATH=/home/sherpa-ncnn/sherpa-ncnn/build/bin:$PATH

在sherap-ncnn中测试：
./decode-file-c-api
$dir/data/lang_bpe_500/tokens.txt
$dir/exp/encoder_jit_trace-pnnx.ncnn.param
$dir/exp/encoder_jit_trace-pnnx.ncnn.bin
$dir/exp/decoder_jit_trace-pnnx.ncnn.param
$dir/exp/decoder_jit_trace-pnnx.ncnn.bin
$dir/exp/joiner_jit_trace-pnnx.ncnn.param
$dir/exp/joiner_jit_trace-pnnx.ncnn.bin
$dir/test_wavs/1089-134686-0001.wav

输出为：
AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT
UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS

7.int8 quantization with sherpa-ncnn

---

./generate-int8-scale-table \ （程序崩溃：Segmentation fault）
$dir/exp/encoder_jit_trace-pnnx.ncnn.param
$dir/exp/encoder_jit_trace-pnnx.ncnn.bin
$dir/exp/decoder_jit_trace-pnnx.ncnn.param
$dir/exp/decoder_jit_trace-pnnx.ncnn.bin
$dir/exp/joiner_jit_trace-pnnx.ncnn.param
$dir/exp/joiner_jit_trace-pnnx.ncnn.bin
./encoder-scale-table.txt
./joiner-scale-table.txt
./wav.txt

wav.txt内容如下：
/home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/test_wavs/1089-134686-0001.wav
/home/models/icefall-asr-librispeech-conv-emformer-transducer-stateless2-2022-07-05/test_wavs/1221-135766-0001.wav

[wanganddi0918]

我尝试将 sherpa-ncnn 的版本进行回退（b55bfb78fef814c320d97b7e79a283853056b3ce）就可以正常跑通了，应该是 master 代码和 generate-int8-scale-table.cc 哪里不适配了，辛苦老师有时间帮忙看下