PyAFQ failure

[araikes]

Summary

Piggybacking off of #122, I ran into errors with PyAFQ (--recon-spec pyqfq_tractometry). This one seems a bit more challenging/problematic to diagnose and I understand that checking on n = 1 isn't super helpful, but it's where we're at for the moment. Essentially, I'm getting an error that there's a NaN response in the CSD fitting. I haven't tested the MRtrix + PyAFQ pipeline (MRtrix MSMT ACT HSVS took 52 hours and just finished), but I know that I get a valid CSD response function from these data based on the other pipelines, which leads me to believe that something isn't getting passed to the node.

Additional details

• QSIRecon version:
• Docker version:
• Singularity version:

What were you trying to do?

Test for unanticipated failures.

What did you expect to happen?

Unanticipated failures

What actually happened?

Unanticipated failure

Reproducing the bug

apptainer run -\
-containall --writable-tmpfs \
-B $PWD/derivatives/qsiprep_0.22.1/qsiprep:/input \
-B $PWD/derivatives/qsirecon:/output \
-B /tmp:/tmp \
-B /groups/adamraikes/license.txt:/license.txt \
$RECON /input /output participant \
--participant_label 001 \
-w /tmp  \
--nthreads 8 --omp-nthreads 8 --fs-license-file /license.txt --recon-spec pyafq_tractometry--input-type qsiprep

Crash log

Node: qsirecon_0_23_wf.sub-001_pyafq_tractometry.sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf.pyafq_tractometry.run_afq
Working directory: /tmp/qsirecon_0_23_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq

Node inputs:

bval_file = <undefined>
bvec_file = <undefined>
dwi_file = <undefined>
itk_file = <undefined>
kwargs = {'directions': 'prob', 'max_angle': 30.0, 'sphere': None, 'seed_mask': None, 'seed_threshold': 0, 'n_seeds': 1, 'random_seeds': False, 'rng_seed': None, 'stop_mask': None, 'stop_threshold': 0, 'step_size': 0.5, 'odf_model': 'CSD', 'tracker': 'local', 'nb_points': False, 'nb_streamlines': False, 'seg_algo': 'AFQ', 'clip_edges': False, 'parallel_segmentation': {'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}, 'progressive': True, 'greater_than': 50, 'rm_small_clusters': 50, 'model_clust_thr': 1.25, 'reduction_thr': 25, 'refine': False, 'pruning_thr': 12, 'b0_threshold': 50, 'prob_threshold': 0, 'roi_dist_tie_break': False, 'dist_to_waypoint': None, 'rng': None, 'return_idx': False, 'presegment_bundle_dict': None, 'presegment_kwargs': {}, 'filter_by_endpoints': True, 'dist_to_atlas': 4, 'save_intermediates': None, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'min_bval': None, 'max_bval': None, 'filter_b': True, 'robust_tensor_fitting': False, 'csd_response': None, 'csd_sh_order': None, 'csd_lambda_': 1, 'csd_tau': 0.1, 'gtol': 0.01, 'brain_mask_definition': None, 'bundle_info': None, 'reg_template_spec': 'mni_T1', 'mapping_definition': None, 'reg_subject_spec': 'power_map', 'profile_weights': 'gauss', 'scalars': ['dti_fa', 'dti_md'], 'import_tract': None, 'sbv_lims_bundles': [None, None], 'volume_opacity_bundles': 0.3, 'n_points_bundles': 40, 'sbv_lims_indiv': [None, None], 'volume_opacity_indiv': 0.3, 'n_points_indiv': 40, 'viz_backend_spec': 'plotly_no_gif', 'virtual_frame_buffer': False, 'omp_nthreads': 8}
mask_file = <undefined>
n_procs = 8
tck_file = <undefined>

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node run_afq.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 335, in csd_params
	    csdf = csd_fit_model(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 67, in _fit
	    return _model(gtab, data, response, sh_order, csd_fa_thr).fit(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 56, in _model
	    raise CsdNanResponseError
	AFQ.models.csd.CsdNanResponseError

	The above exception was the direct cause of the following exception:

	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/interfaces/pyafq.py", line 114, in _run_interface
	    myafq.export_all()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 202, in export_all
	    export_all_helper(self, seg_algo, xforms, indiv, viz)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/utils.py", line 121, in export_all_helper
	    api_afq_object.export("template_xform")
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 155, in export
	    return self.wf_dict[section][attr_name]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
	    args.append(opts[name])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
	    return self._examine_val(k, ps.PMap.__getitem__(self, k))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
	    val = val()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
	    def curry_choice(k, args): return lambda:choose_fn(k, args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
	    return vs[-1][k]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
	    args.append(opts[name])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
	    return self._examine_val(k, ps.PMap.__getitem__(self, k))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
	    val = val()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
	    def curry_choice(k, args): return lambda:choose_fn(k, args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
	    return vs[-1][k]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
	    result = self.function(*args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/mapping.py", line 182, in get_reg_subject
	    reg_subject_spec = data_imap[filename_dict[reg_subject_spec]]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
	    args.append(opts[name])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
	    return self._examine_val(k, ps.PMap.__getitem__(self, k))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
	    val = val()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
	    def curry_choice(k, args): return lambda:choose_fn(k, args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
	    return vs[-1][k]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
	    result = self.function(*args)
	  File "<string>", line 2, in wrapper_has_args_func
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 149, in wrapper_as_file
	    gen, meta = func(*args[:og_arg_count], **kwargs)
	  File "<string>", line 2, in wrapper_has_args_func
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 220, in wrapper_as_img
	    data, meta = func(*args[:og_arg_count], **kwargs)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 342, in csd_params
	    raise CsdNanResponseError(
	AFQ.models.csd.CsdNanResponseError: Could not compute CSD response function for file: 
	<class 'nibabel.nifti1.Nifti1Image'>
	data shape (161, 198, 171, 137)
	affine:
	[[ -1.   0.   0.  80.]
	 [  0.  -1.   0.  81.]
	 [  0.   0.   1. -80.]
	 [  0.   0.   0.   1.]]
	metadata:
	<class 'nibabel.nifti1.Nifti1Header'> object, endian='<'
	sizeof_hdr      : 348
	data_type       : b''
	db_name         : b''
	extents         : 0
	session_error   : 0
	regular         : b''
	dim_info        : 0
	dim             : [  4 161 198 171 137   1   1   1]
	intent_p1       : 0.0
	intent_p2       : 0.0
	intent_p3       : 0.0
	intent_code     : none
	datatype        : float64
	bitpix          : 64
	slice_start     : 0
	pixdim          : [1. 1. 1. 1. 1. 1. 1. 1.]
	vox_offset      : 0.0
	scl_slope       : nan
	scl_inter       : nan
	slice_end       : 0
	slice_code      : unknown
	xyzt_units      : 0
	cal_max         : 0.0
	cal_min         : 0.0
	slice_duration  : 0.0
	toffset         : 0.0
	glmax           : 0
	glmin           : 0
	descrip         : b''
	aux_file        : b''
	qform_code      : unknown
	sform_code      : aligned
	quatern_b       : 0.0
	quatern_c       : 0.0
	quatern_d       : 1.0
	qoffset_x       : 80.0
	qoffset_y       : 81.0
	qoffset_z       : -80.0
	srow_x          : [-1.  0.  0. 80.]
	srow_y          : [ 0. -1.  0. 81.]
	srow_z          : [  0.   0.   1. -80.]
	intent_name     : b''
	magic           : b'n+1'
	.

[araikes]

Running this on the current state of #131. Crash is the same but now populated with file paths:

Node: qsirecon_0_0_wf.sub-001_pyafq_tractometry.sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf.pyafq_tractometry.run_afq
Working directory: /output/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq

Node inputs:

bval_file = /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi.bval
bvec_file = /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi.bvec
dwi_file = /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi.nii.gz
itk_file = /input/qsiprep/sub-001/anat/sub-001_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
kwargs = {'directions': 'prob', 'max_angle': 30.0, 'sphere': None, 'seed_mask': None, 'seed_threshold': 0, 'n_seeds': 1, 'random_seeds': False, 'rng_seed': None, 'stop_mask': None, 'stop_threshold': 0, 'step_size': 0.5, 'odf_model': 'CSD', 'tracker': 'local', 'nb_points': False, 'nb_streamlines': False, 'seg_algo': 'AFQ', 'clip_edges': False, 'parallel_segmentation': {'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}, 'progressive': True, 'greater_than': 50, 'rm_small_clusters': 50, 'model_clust_thr': 1.25, 'reduction_thr': 25, 'refine': False, 'pruning_thr': 12, 'b0_threshold': 50, 'prob_threshold': 0, 'roi_dist_tie_break': False, 'dist_to_waypoint': None, 'rng': None, 'return_idx': False, 'presegment_bundle_dict': None, 'presegment_kwargs': {}, 'filter_by_endpoints': True, 'dist_to_atlas': 4, 'save_intermediates': None, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'min_bval': None, 'max_bval': None, 'filter_b': True, 'robust_tensor_fitting': False, 'csd_response': None, 'csd_sh_order': None, 'csd_lambda_': 1, 'csd_tau': 0.1, 'gtol': 0.01, 'brain_mask_definition': None, 'bundle_info': None, 'reg_template_spec': 'mni_T1', 'mapping_definition': None, 'reg_subject_spec': 'power_map', 'profile_weights': 'gauss', 'scalars': ['dti_fa', 'dti_md'], 'import_tract': None, 'sbv_lims_bundles': [None, None], 'volume_opacity_bundles': 0.3, 'n_points_bundles': 40, 'sbv_lims_indiv': [None, None], 'volume_opacity_indiv': 0.3, 'n_points_indiv': 40, 'viz_backend_spec': 'plotly_no_gif', 'virtual_frame_buffer': False, 'omp_nthreads': 8}
mask_file = /output/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-001_desc-brain_mask_trans.nii.gz
n_procs = 8
tck_file = <undefined>

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node run_afq.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 335, in csd_params
	    csdf = csd_fit_model(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 67, in _fit
	    return _model(gtab, data, response, sh_order, csd_fa_thr).fit(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 56, in _model
	    raise CsdNanResponseError
	AFQ.models.csd.CsdNanResponseError

	The above exception was the direct cause of the following exception:

	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/interfaces/pyafq.py", line 114, in _run_interface
	    myafq.export_all()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 202, in export_all
	    export_all_helper(self, seg_algo, xforms, indiv, viz)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/utils.py", line 121, in export_all_helper
	    api_afq_object.export("template_xform")
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 155, in export
	    return self.wf_dict[section][attr_name]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
	    args.append(opts[name])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
	    return self._examine_val(k, ps.PMap.__getitem__(self, k))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
	    val = val()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
	    def curry_choice(k, args): return lambda:choose_fn(k, args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
	    return vs[-1][k]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
	    args.append(opts[name])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
	    return self._examine_val(k, ps.PMap.__getitem__(self, k))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
	    val = val()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
	    def curry_choice(k, args): return lambda:choose_fn(k, args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
	    return vs[-1][k]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
	    result = self.function(*args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/mapping.py", line 182, in get_reg_subject
	    reg_subject_spec = data_imap[filename_dict[reg_subject_spec]]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
	    args.append(opts[name])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
	    return self._examine_val(k, ps.PMap.__getitem__(self, k))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
	    val = val()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
	    def curry_choice(k, args): return lambda:choose_fn(k, args)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
	    return vs[-1][k]
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
	    self._run_node(self.plan.efferents[k])
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
	    if not found: res = node(self)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
	    result = self.function(*args)
	  File "<string>", line 2, in wrapper_has_args_func
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 149, in wrapper_as_file
	    gen, meta = func(*args[:og_arg_count], **kwargs)
	  File "<string>", line 2, in wrapper_has_args_func
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 220, in wrapper_as_img
	    data, meta = func(*args[:og_arg_count], **kwargs)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 342, in csd_params
	    raise CsdNanResponseError(
	AFQ.models.csd.CsdNanResponseError: Could not compute CSD response function for file: 
	<class 'nibabel.nifti1.Nifti1Image'>
	data shape (161, 198, 171, 137)
	affine:
	[[ -1.   0.   0.  80.]
	 [  0.  -1.   0.  81.]
	 [  0.   0.   1. -80.]
	 [  0.   0.   0.   1.]]
	metadata:
	<class 'nibabel.nifti1.Nifti1Header'> object, endian='<'
	sizeof_hdr      : 348
	data_type       : b''
	db_name         : b''
	extents         : 0
	session_error   : 0
	regular         : b''
	dim_info        : 0
	dim             : [  4 161 198 171 137   1   1   1]
	intent_p1       : 0.0
	intent_p2       : 0.0
	intent_p3       : 0.0
	intent_code     : none
	datatype        : float64
	bitpix          : 64
	slice_start     : 0
	pixdim          : [1. 1. 1. 1. 1. 1. 1. 1.]
	vox_offset      : 0.0
	scl_slope       : nan
	scl_inter       : nan
	slice_end       : 0
	slice_code      : unknown
	xyzt_units      : 0
	cal_max         : 0.0
	cal_min         : 0.0
	slice_duration  : 0.0
	toffset         : 0.0
	glmax           : 0
	glmin           : 0
	descrip         : b''
	aux_file        : b''
	qform_code      : unknown
	sform_code      : aligned
	quatern_b       : 0.0
	quatern_c       : 0.0
	quatern_d       : 1.0
	qoffset_x       : 80.0
	qoffset_y       : 81.0
	qoffset_z       : -80.0
	srow_x          : [-1.  0.  0. 80.]
	srow_y          : [ 0. -1.  0. 81.]
	srow_z          : [  0.   0.   1. -80.]
	intent_name     : b''
	magic           : b'n+1'
	.

[araikes]

From the terminal output, this immediately precedes the error:

INFO:AFQ:No seed mask given, using FA (or first scalar if none are FA)thresholded to 0.2
INFO:AFQ:No stop mask given, using FA (or first scalar if none are FA)thresholded to 0.2
INFO:AFQ:Saving /xdisk/adamraikes/PD_allo/derivatives/qsirecon_unstable_v2/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq/study/subject/PYAFQ/sub-001_ses-01_dir-AP_run-001_desc-brainmasktrans_dwi.nii.gz
INFO:AFQ:Saving /xdisk/adamraikes/PD_allo/derivatives/qsirecon_unstable_v2/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_02_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq/study/subject/PYAFQ/sub-001_ses-02_dir-AP_run-001_desc-brainmasktrans_dwi.nii.gz
WARNING:AFQ:Failed to export warped b0. This could be because your mapping type is only compatible with transformation from template to subject space. The error is: Could not compute CSD response function for file: 
<class 'nibabel.nifti1.Nifti1Image'>

[smeisler]

I wonder if this is due to an older pyafq version? I remember this, or a similar error, being discussed here: yeatmanlab/pyAFQ#1156

[araikes]

So possibly related... (and meant to post this on Friday but apparently didn't actually hit the button)...

multishell_scalarfest yields a PeaksReport failure, also related to ODF nans:

Cmdline:
	recon_plot --background_image /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_dwiref.nii.gz --directions /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_dirs.npy --mask_file /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-001_desc-brain_mask_trans.nii.gz --amplitudes /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_amp.nii.gz --odfs_image odfs_mosaic.png --odf_rois /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_dwi_specific_anat_wf/odf_rois/crossing_rois_trans.nii.gz --peaks_image peaks_mosaic.png
Stdout:
	241025-15:42:08,45 nipype.interface INFO:
		 loading amplitudes=/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_amp.nii.gz, directions=/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_dirs.npy to plot ODF/peaks
	241025-15:42:32,401 nipype.interface INFO:
		 saving peaks image to /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/plot_peaks/peaks_mosaic.png
	241025-15:42:32,591 nipype.interface INFO:
		 Plotting slice indices {'x': [73, 80, 87], 'y': [90, 99, 107], 'z': [78, 85, 92]}
Stderr:
	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/bin/recon_plot", line 8, in <module>
	    sys.exit(recon_plot())
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 129, in recon_plot
	    peak_slice_series(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 262, in peak_slice_series
	    plot_peak_slice(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 192, in plot_peak_slice
	    peak_dirs, peak_values = peaks_from_odfs(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 335, in peaks_from_odfs
	    direction, pk, ind = peak_directions(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/dipy/direction/peaks.py", line 132, in peak_directions
	    values, indices = local_maxima(odf, sphere.edges)
	  File "recspeed.pyx", line 249, in dipy.reconst.recspeed.local_maxima
	ValueError: odf can not have nans
Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
	    setattr(outputs, key, val)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
	    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
	    self.error(objekt, name, str(value))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
	    raise TraitError(
	traits.trait_errors.TraitError: The 'peak_report' trait of a _ReconPeaksReportOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/plot_peaks/peaks_mosaic.png' <class 'str'> was specified.

	During handling of the above exception, another exception occurred:

	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 400, in run
	    outputs = self.aggregate_outputs(runtime)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs
	    raise FileNotFoundError(msg)
	FileNotFoundError: No such file or directory '/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/plot_peaks/peaks_mosaic.png' for output 'peak_report' of a CLIReconPeaksReport interface

So PyAFQ and MAPMRI, which both run DIPY to generate the ODFs, fail due to nans. By contrast, MRtrix has no issue estimating FODs for either the connectivity pipelines or PyAFQ.

[mattcieslak]

I wonder if we should drop multishell_scalarfest from the official recon workflows. It's meant to be more of a showcase for the different recon methods and probably shouldn't be encouraged for hypothesis-driven research. WDYT @araikes @smeisler ?

[smeisler]

I personally am in favor of keeping it, because (1) sometimes you just want that much stuff (2) it gives users a hint about how to create custom recon specs that do several things (3) it saves people from inadvertently using wrong NODDI coefficients in gray matter.