Guppy initialization fails, unexplicably

[cvermeulen88]

Yesterday megalodon stopped working, and now consistently gives the error:

ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.

However when I run the guppy server initialization command in the log file, it starts up just fine.

These are the commands:

megalodon \
    example_data/ \
    --guppy-params "-d ~/asmetha/rerio/basecall_models/" \
    --write-mods-text \
    --guppy-config res_dna_r941_min_modbases_5mC_CpG_v001.cfg \
    --outputs mods \
    --reference ~/compdir/refgenomes/chm13_V2/chm13v2.0.fa \
    --mod-motif m CG 0 \
    --devices 0\
    --processes 10 \
    --guppy-server-path ~/compdir/guppy/v6.3/ont-guppy/bin/guppy_basecall_server \
    --output-directory megalodon_output_rerio_baseenv 

Output:

[16:39:43] Running Megalodon version 2.5.0
******************** WARNING: "mods" output requested, so "per_read_mods" will be added to outputs. ********************
[16:39:43] Loading guppy basecalling backend
****************************************************************************************************
        ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.
                Try running the guppy server initialization command found in log.txt in order to pinpoint the source of this issue.
****************************************************************************************************

This is the command in the log.txt:

/home/cog/cvermeulen/compdir/guppy/v6.3/ont-guppy/bin/guppy_basecall_server -p auto -l megalodon_output_rerio_baseenv/guppy_log -c res_dna_r941_min_modbases_5mC_CpG_v001.cfg --quiet -x cuda:0 -d ~/asmetha/rerio/basecall_models/

This works, with only output:
CRASHPAD MESSAGE:

And when I remove the --quiet tag:

CRASHPAD MESSAGE:
ONT Guppy basecall server software version 6.3.8+d9e0f64, client-server API version 13.0.0, minimap2 version 2.22-r1101
log path:            megalodon_output_rerio_baseenv/guppy_log
chunk size:          2000
chunks per runner:   512
max queued reads:    2000
num basecallers:     4
num socket threads:  2
max returned events: 256000
gpu device:          cuda:0
kernel path:
runners per device:  4
Use of this software is permitted solely under the terms of the end user license agreement (EULA).By running, copying or accessing this software, you are demonstrating your acceptance of the EULA.
The EULA may be found in /hpc/compgen/users/cvermeulen/guppy/v6.3/ont-guppy/bin



Config loaded:
config file:               /home/cog/cvermeulen/asmetha/rerio/basecall_models/res_dna_r941_min_modbases_5mC_CpG_v001.cfg
model file:                /home/cog/cvermeulen/asmetha/rerio/basecall_models/res_dna_r941_min_modbases_5mC_CpG_v001.jsn
model version id           None
adapter scaler model file: None
Starting server on port: ipc:///scratch/16277515/bb8a-9f15-179a-4d44

I tried a lot (new conda environments, reinstalling megalodon, different guppy versions), but I really can't figure out why it isn't working, so any help is really appreciated.

[Arkadiy-Garber]

Megalodon version: 2.5.0
(megalodon_env) MAB@Axceleron-WKS:~/2748_NP_methylation/fast5_pass$ megalodon barcode12/ --guppy-config /data/MAB/bin/ont-guppy-cpu/data/dna_r10.3_450bps_fast.cfg --remora-modified-bases dna_r10.4.1_e8 fast 4.0.0 5hmc_5mc CG 2 --outputs basecalls mappings mod_mappings mods --reference /data/MAB/2748_NP_methylation/fastq_pass/barcode12/assembly/assembly.fasta --processes 24 --guppy-server-path /data/MAB/bin/ont-guppy-cpu/bin/guppy_basecaller --overwrite
[16:37:13] Running Megalodon version 2.5.0
******************** WARNING: "mods" output requested, so "per_read_mods" will be added to outputs. ********************
[16:37:13] Loading guppy basecalling backend
****************************************************************************************************
	ERROR: Guppy server initialization failed. See guppy logs in [--output-directory] for more details.
		Try running the guppy server initialization command found in log.txt in order to pinpoint the source of this issue.
****************************************************************************************************

from the log:

(megalodon_env) MAB@Axceleron-WKS:~/2748_NP_methylation/fast5_pass/megalodon_results$ /data/MAB/bin/ont-guppy-cpu/bin/guppy_basecaller -p auto -l megalodon_results/guppy_log -c /data/MAB/bin/ont-guppy-cpu/data/dna_r10.3_450bps_fast.cfg --quiet
Unexpected token '-p' on command-line.
Unexpected token 'auto' on command-line.
Try 'guppy_basecaller --help' for more information.
(megalodon_env) MAB@Axceleron-WKS:~/2748_NP_methylation/fast5_pass/megalodon_results$