Docker Image #15
Comments
|
Sorry, no we don’t have a docker image yet.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: afarrell@genetics.utah.edu
http://marthlab.org/
=================================
… On Jan 30, 2020, at 8:36 AM, ksarathbabu ***@***.***> wrote:
Is there a Docker image available for RUFUS?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Haven't heard back about building from source for two weeks now #18, so I've tried to build a Docker image. I'm getting errors related to
|
|
Sorry I missed that email, things have been crazy here with Covid stuff. The build definitely needs the internet sorry. We pull allot of the dependencies straight from GitHub when we’re building. I would recommend just building on a similar system with internet connection, then zipping the whole RUFUS dir and moving it to the secure system.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: afarrell@genetics.utah.edu
http://marthlab.org/
=================================
… On Jan 28, 2021, at 9:33 PM, Matthew J. Oldach ***@***.***> wrote:
Seeking outside expertise: https://stackoverflow.com/questions/65948810/cmake-error-the-source-directory-does-not-appear-to-contain-cmakelists-txt-for
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Can I use a version of |
|
Greater than 4.9.2 should be fine but I haven’t tested it extensively.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: afarrell@genetics.utah.edu
http://marthlab.org/
=================================
… On Jan 29, 2021, at 5:48 PM, Matthew J. Oldach ***@***.***> wrote:
I would recommend just building on a similar system with internet connection, then zipping the whole RUFUS dir and moving it to the secure system.
Can I use a version of gcc > 4.9.2? Or, can it only be built on gcc-4.9.2?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
I have created a Dockerfile for RUFUS as pull request #20. I had no issues with the build process when using the dependencies outlined in the README. |
Hi @kohrar I really appreciate the attempt. Could you please provide some instructions for how you would run it ( I pulled #20, built a container from your Next, I build a Then I Finally, I try running the following command: However, I'm getting the following error: |
|
Hi Matthew, I've only done a cursory test with RUFUS after building the image and then running it with Regarding your issue, it looks like you're running RUFUS from a bind mount at Did you copy the entire |
So this part of my call was wrong: However, when I change it to the following I still get an error:
It's almost like the I cannot see inside the container? Let's take a look inside the Can see contents of container in
|
|
One issue is that I need to supply However, I'm getting a |
|
Hi @kohrar
Did you not try and run I've tried the following but there appears to be issues: |
|
you need to be in the directory RUFUS/testRun/ to run the test. Sorry about that, I should make that clearer in the documentation.
=====================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: JAndrewRFarrell@gmail.com
http://marthlab.org/ <http://marthlab.org/>=====================================
… On Feb 17, 2021, at 10:40 AM, Matthew J. Oldach ***@***.***> wrote:
Hi @kohrar <https://github.com/kohrar>
I've only done a cursory test with RUFUS after building the image and then running it with docker run --rm -ti rufus bash
Did you not try and run /testRun/runTest.sh with Docker to verify that the Docker contain works?
I've tried the following but there appears to be issues:
(base) ***@***.***:~/DOCKER-CONTAINERS/RUFUS$ sudo docker run --rm -it rufus-v1.0
***@***.***:/# ls
RUFUS boot etc lib media opt root sbin sys usr
bin dev home lib64 mnt proc run srv tmp var
***@***.***:/# bash RUFUS/testRun/runTest.sh
RUFUS/testRun/runTest.sh: line 1: ./../runRufus.sh: No such file or directory
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHXUHOKRGRCYJS5C6MDS7P5QHANCNFSM4KNYDGCQ>.
|
|
Hi @moldach, I ran the runTest script without issue within the Singularity image. I am not mounting an external version of RUFUS into the image as you are doing, which I think is leading to your issues. See my usage below: Regarding your library issue, this is what should be loaded. Everything is within the image and not from some external bind mount. |
|
Hello @kohrar thank you very much for providing that detailed explanation and reproducible example!
I'm fairly new to using Singularity and in the past, for example with So, the fact that As a word of caution for others, running the test on the singularity container failed for me; however - and luckily - it does work on my data 🥳 Working solutionError on
|
|
I see a couple of errors going on you’ll want to fix. There seems to be an error with your perl install, Ive never seen these errors and not sure what could be causing them. But RUFUS also appears to go right past them and work just fine, the down street steps seem to be ok. The real problem thats killing your run is you seem to have an issues with samtools. RUFUS checks that samtools is there and it appears be to be, but when calling samtools sort your getting an error “sort: invalid option —’T’” and “sort: invalid options ‘O’”. Ive seen this happen when using a very old version of samtools or when you don’t have samtools installed. Check and make sure when you type “samtools” on the command line that it runs and check the version that you have installed.
Perl error:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
=====================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: JAndrewRFarrell@gmail.com
http://marthlab.org/
=====================================
… On Feb 18, 2021, at 8:58 AM, Matthew J. Oldach ***@***.***> wrote:
Hello @kohrar thank you very much for providing that detailed explanation and reproducible example!
I am not mounting an external version of RUFUS into the image as you are doing
-H $PWD turned out to be critical for this.
I'm fairly new to using Singularity and in the past, for example with Manta (and DeepVariant) I was using -B $PWD when faced with a function parameter that asked for --runDir:
singularity exec \
-B /project/M-mtgraovac182840/matthew/tool-testing/MTG_human_genomics_pipeline-master/alignment/bwa/:/bams,/project/M-mtgraovac182840/indexes/GRCh37/:/reference \
-B $PWD \
/project/M-mtgraovac182840/tools/manta-1.6.0.img \
/manta/bin/configManta.py \
--bam /bams/proband_bwaMEM_sort_dedupped.bam \
--referenceFasta /reference/Homo_sapiens.GRCh37.dna.toplevel.fa \
--runDir $PWD
So, the fact that RUFUS doesn't ask for a output directory (and instead prints to the $PWD) caused the -B $PWD solution to fail.
As a word of caution for others, running the test on the singularity container failed for me; however - and luckily - it does work on my data 🥳
Working solution
singularity -s exec \
-B /project/M-mtgraovac182840/matthew/tool-testing/MTG_oldPipeScript/alignment/bwa/:/usr/lib/locale/ \
-B /project/M-mtgraovac182840/indexes/GRCh37/:/usr/lib/locale/index/ \
-H `pwd` \
/project/M-mtgraovac182840/tools/rufus.sif \
./RUFUS/runRufus.sh \
-s /usr/lib/locale/proband_bwaMEM_sort_dedupped.bam \
-c /usr/lib/locale/mom_bwaMEM_sort_dedupped.bam \
-c /usr/lib/locale/dad_bwaMEM_sort_dedupped.bam \
-t 2 \
--kmersize 25 \
--ref=/usr/lib/locale/index/Homo_sapiens.GRCh37.dna.toplevel.fa
Error on testRun
## test asks for 40 cores but we will just ask for 30
$ salloc --time=0:30:0 --mem-per-cpu=5000 --cpus-per-task=30
$ singularity run -H `pwd` rufus.sif
Singularity> cd /
Singularity> cp -r RUFUS /tmp
Singularity> cd /tmp/RUFUS/testRun/
Singularity> sh runTest.sh
checking for samtools
/usr/bin/samtools
samtools found
_arg_fastqA =
_arg_fastqB =
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Final reference path being used is /tmp/RUFUS/testRun/../resources/references/small_test_human_reference_v37_decoys.fa
Final bwa reference path being used is /tmp/RUFUS/testRun/../resources/references/small_test_human_reference_v37_decoys.fa
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
proband extension is bam
you provided the proband cram file /tmp/RUFUS/testRun/Child.bam
parent file name is Mother.bam
parent file extension name is bam
You provided the control bam file /tmp/RUFUS/testRun/Mother.bam
parent file name is Father.bam
parent file extension name is bam
You provided the control bam file /tmp/RUFUS/testRun/Father.bam
~~~~~~~~~~~~ printing out paramater values used in script ~~~~~~~~~~~~~~~~
value of ProbandGenerator Child.bam.generator
Value of ParentGenerators:
Mother.bam.generator
Father.bam.generator
Value of K is: 25
Value of Threads is: 40
value of ref is: /tmp/RUFUS/testRun/../resources/references/small_test_human_reference_v37_decoys.fa
value of min is:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Did not provide refHash
$_arg_min is empty
_arg_min is
MutantMinCov is
parent is Mother.bam.generator
parent is Father.bam.generator
Running jellyfish for Mother.bam.generator
Running jellyfish for Father.bam.generator
Running jellyfish for Child.bam.generator
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
min not provided, building model
staring model
Call is histoFile HS ReadLength Threads
Parent File open - Child.bam.generator.Jhash.histo
first line = 0 - 0
getting another
got 1 - 0
getting another
got 2 - 1089
going with 2 - 1089
Number of reads = 3630
I = 0 0
I = 1 1089
I = 2 107
I = 3 31
I = 4 44
I = 5 21
I = 6 27
I = 7 54
I = 8 129
I = 9 98
SC = 25 vlaue = 1135
stdi = 31 stdev = 6
best error is 1/x^3.34726
On 1 pass
best Factor = 0 steps = 12
bestSC = 27.4356 steps = 4
best StdDev = 6.44494 steps = 3
best skew factor = 0 steps = 0
best Power factor = 1 steps = 3
On 2 pass
best Factor = 0 steps = 12
bestSC = 26.9935 steps = 5
best StdDev = 6.36922 steps = 3
best skew factor = 0 steps = 0
best Power factor = 1 steps = 3
On 3 pass
best Factor = 0 steps = 12
bestSC = 27.0027 steps = 5
best StdDev = 6.6012 steps = 3
best skew factor = 0 steps = 0
best Power factor = 1 steps = 3
Best Model is SC = 27.0027 StdDev = 6.6012 F = 0 skew = 0 bestP = 1
GenomeSize = 31411.6
prob not error = 0.0116117
prob not error = 0.136157
prob not error = 0.441105
prob not error = 0.722617
prob not error = 0.871386
prob not error = 0.937632
this one
GenomeSize = 31411.6
Inflection point = 3
Recomended RUFUS cutoff = -6.00331
-1std = 20.4015 -2std = 13.8003 -3std = 7.1991 -4std = 0.597893
done with model
mutant min coverage from generated model is 5
mutant SC coverage from generated model is 25
MaxHashDepth = 125
made it
made it here
starting RUFUS filter
_arg_fastqA =
_arg_fastqB =
running this one
Call is PreBuiltMutHash Mutant.Mate1.fq Mutant.Mate2.fq firstpassfile hashsize MinQ HashCountThreshold threads
VM: 19756; RSS: 2564
Paramaters are:
PreBuiltMutHash = Child.bam.generator.k25_c5.HashList
Mutant.mate1.fq = Child.bam.generator.temp.mate1.fastq
Mutant.mate2.fq = Child.bam.generator.temp.mate2.fastq
out stub = Child.bam.generator
HashSize = 25
MinQ = 13
HashCountThreshold = 1
Threads = 38
Parent File open - Child.bam.generator.k25_c5.HashList
MutFile.mate1 is Child.bam.generator.temp.mate1.fastq
here
##File Opend
MutFile.mate2 is Child.bam.generator.temp.mate2.fastq
##File Opend
Reading in pre-built hash talbe
starting
Reading in MutHashFile
Done Hash Files
Mutations Hash size is 74
I am using 2564
VM: 19756; RSS: 2564; maxVM: 19756; maxRSS: 2564
Starting Search
Read in 2040 lines: Found 20 Reads per sec = 6.16048e-11
Done running RUFUS.Filter.cpp
skipping fastp fix
sort: invalid option -- 'T'
sort: invalid option -- 'O'
samblaster: Version 0.1.26
samblaster: Inputting from stdin
samblaster: Outputting to stdout
open: No such file or directory
[bam_sort_core] fail to open file Child.bam.generator.Mutations.fastq
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 40 sequences (6040 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 20, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (274, 298, 334)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (154, 454)
[M::mem_pestat] mean and std.dev: (297.37, 36.62)
[M::mem_pestat] low and high boundaries for proper pairs: (94, 514)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 40 reads in 0.009 CPU sec, 0.004 real sec
samblaster: Loaded 2 header sequence entries.
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM file.
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[main_samview] fail to read the header from "Child.bam.generator.Mutations.fastq.bam".
ERROR: BWA failed on Child.bam.generator.Mutations.fastq. Either the files are exactly the same of something went wrong in previous step
## Not empty files, just really small....
Singularity> ls -sh
total 11M
368K Child.bam 0 Child.bam.generator.temp.mate2.fastq
4.0K Child.bam.generator 272K Father.bam
4.0K Child.bam.generator.Jelly.chr 4.0K Father.bam.generator
200K Child.bam.generator.Jhash 4.0K Father.bam.generator.Jelly.chr
68K Child.bam.generator.Jhash.histo 188K Father.bam.generator.Jhash
9.1M Child.bam.generator.Jhash.histo.7.7.dist 68K Father.bam.generator.Jhash.histo
12K Child.bam.generator.Jhash.histo.7.7.model 364K Mother.bam
0 Child.bam.generator.Jhash.histo.7.7boom.prob 4.0K Mother.bam.generator
8.0K Child.bam.generator.Mutations.Mate1.fastq 4.0K Mother.bam.generator.Jelly.chr
8.0K Child.bam.generator.Mutations.Mate2.fastq 192K Mother.bam.generator.Jhash
0 Child.bam.generator.Mutations.fastq.bam 68K Mother.bam.generator.Jhash.histo
4.0K Child.bam.generator.filter.chr 4.0K clean.sh
4.0K Child.bam.generator.k25_c5.HashList 4.0K mer_counts_merged.jf
0 Child.bam.generator.temp 4.0K runDevTest.sh
0 Child.bam.generator.temp.mate1.fastq 4.0K runTest.sh
Singularity> exit
Thanks again for all the help!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
So the job ran for 24 hours but failed:
What isn't clear to me is which Let's check: If I look for the paths of So it's clear that this is an issue with the tools inside the I'm surprised that
|
|
Yup, your Samtools isn’t working properly, you either don’t have it installed or you have a very old version. I’ve seen that invalid option T and O before when Sam tools isn’t installed. Rufus checks for samtools and your system passes but for some reason samtools sort is still failing. Check that samtools runs when you type “samtools”, check the version of samtools, and check when you type “samtools sort -h” that the O and T options are there.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: afarrell@genetics.utah.edu
http://marthlab.org/
=================================
… On Feb 19, 2021, at 9:26 AM, Matthew J. Oldach ***@***.***> wrote:
So the job ran for 24 hours but failed: Slurm Job_id=2711 Name=rufus_test Ended, Run time 1-00:08:45, COMPLETED, ExitCode 0
sort: invalid option -- 'T'
sort: invalid option -- 'O'
open: No such file or directory
[bam_sort_core] fail to open file proband_bwaMEM_sort_dedupped.bam.generator.Mutations.fastq
samblaster: Version 0.1.26
samblaster: Inputting from stdin
samblaster: Outputting to stdout
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 2000000 sequences (300000000 bp)...
[M::process] read 2000000 sequences (300000000 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (22, 416954, 29, 16)
[M::mem_pestat] analyzing insert size distribution for orientation FF...
[M::mem_pestat] (25, 50, 75) percentile: (411, 973, 1348)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3222)
[M::mem_pestat] mean and std.dev: (830.10, 447.09)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 4159)
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (397, 461, 534)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (123, 808)
[M::mem_pestat] mean and std.dev: (467.16, 106.23)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 945)
[M::mem_pestat] analyzing insert size distribution for orientation RF...
[M::mem_pestat] (25, 50, 75) percentile: (405, 1253, 1870)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 4800)
[M::mem_pestat] mean and std.dev: (1221.37, 1039.51)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 6265)
[M::mem_pestat] analyzing insert size distribution for orientation RR...
[M::mem_pestat] (25, 50, 75) percentile: (534, 950, 1892)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 4608)
[M::mem_pestat] mean and std.dev: (1187.62, 956.20)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 5966)
[M::mem_pestat] skip orientation FF
[M::mem_pestat] skip orientation RF
[M::mem_pestat] skip orientation RR
[M::mem_process_seqs] Processed 2000000 reads in 1126.856 CPU sec, 41.206 real sec
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM file.
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[main_samview] fail to read the header from "proband_bwaMEM_sort_dedupped.bam.generator.Mutations.fastq.bam".
There seems to be an error with your perl install, ... The real problem thats killing your run is you seem to have an issues with samtools.
What isn't clear to me is which perl & samtools versions is it trying to use - is it tools from within the RUFUS container or is trying to use versions I installed on my system?
Let's check:
$ singularity run \
-H `pwd` \
/project/M-mtgraovac182840/tools/rufus.sif perl --version
This is perl 5, version 22, subversion 1 (v5.22.1) built for x86_64-linux-gnu-thread-multi
$ singularity run \
-H `pwd` \
/project/M-mtgraovac182840/tools/rufus.sif samtools
Program: samtools (Tools for alignments in the SAM format)
Version: 0.1.19-96b5f2294a
If I look for the paths of samtools and perl on my local host I see a different version:
$ samtools
Program: samtools (Tools for alignments in the SAM format)
Version: 1.3.1 (using htslib 1.3.1)
(base) ***@***.*** RUFUS-TEST]$ perl -v
This is perl 5, version 32, subversion 0 (v5.32.0) built for x86_64-linux
So it's clear that this is an issue with the tools inside the Dockerfile that @kohrar created:
#
# A Dockerfile to get RUFUS running
#
# GCC 4.9 only available up to 16.04
FROM ubuntu:16.04
ARG DEBIAN_FRONTEND=noninteractive
COPY . /RUFUS
RUN set -ex; \
# Dependencies
BUILD_DEPS="cmake build-essential g++-4.9 zlib1g-dev libbz2-dev libbz2-dev liblzma-dev libncurses5-dev"; \
apt-get update; \
apt-get install -y software-properties-common; \
add-apt-repository ppa:ubuntu-toolchain-r/test; \
apt-get install -y python wget git bc $BUILD_DEPS; \
# Build
mkdir -p /RUFUS/bin; \
cd /RUFUS/bin; \
cmake ../ -DCMAKE_C_COMPILER=$(which gcc) -DCMAKE_CXX_COMPILER=$(which g++); \
make; \
# Cleanup
apt-get purge -y --auto-remove $BUILD_DEPS; \
apt-get clean; \
echo done
# Runtime tools
RUN set -ex; \
apt install samtools; \
echo done
I'm surprised that sudo apt install samtools is installing such an old version?
Which version of samtools will work?
could the fact that the base image is Ubuntu 16.04 be why its installing an older version? How to specify which samtools for the container instead?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Correct, as I showed in my above post the following command in @kohrar's I tried to make changes to the And, despite the fact that I included Not sure if there is a more logical/timely way of testing for the presence of |
|
https://pubmed.ncbi.nlm.nih.gov/23165927/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5903934/
=====================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
USTAR Center for Genetic Discovery
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: JAndrewRFarrell@gmail.com
http://marthlab.org/
=====================================
|
|
Any update on this @jandrewrfarrell ? (@kohrar) |
|
Hi @moldach, I have updated the Dockerfile under my pull request (#20) to include the missing dependency required by some RUFUS binaries as well as the newest version of Samtools. This should help with some of the issues you reported. Could you please see if this gets you any further? |
|
I have been experimenting with getting RUFUS operational in some form. I ran into build errors trying to build the source, so decided to use @moldach's docker image, moldach686/rufus-v1.0. Running in Singularity, I get the following error running the test script: Singularity> sh runTest.sh Did not provide refHash I also tried running this on our own data and got different errors depending on which shell I used: with bash: There is a properly generated .sa file in the same directory as the .fa file; I even tried renaming it "ef.sa" to no avail. Anyone have any ideas as to what is causing these errors? Is RUFUS still being actively maintained? Our lab would really like to be able to use it. |
|
The first error your getting is from jellyfish "/tmp/RUFUS/scripts/RunJellyForRUFUS.sh: line 34: 62244 Killed $JELLYFISH count --disk -m $K -L $L -s 8G -t $T -o $GEN.Jhash -C $GEN.fq”. I believe this is due to jellyfish running out of memory, how much memory does your singulatiry instance have available? By default I have RUFUS setup to use at least 32G but in the scripts you could edit jellyfish to use less ram if you need to.
The “bad substitution error” from your second run looks like it's like an issues with different versions of bash compatibility. That line is:
RDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )”
which sets RDIR to the path were runRufus.sh lives. As a workaround you could just hardcode RDIR as the path to the Rufus directory to see if it works.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: ***@***.***
http://marthlab.org/
=================================
… On Oct 18, 2022, at 12:11 PM, Antares ***@***.***> wrote:
I have been experimenting with getting RUFUS operational in some form. I ran into build errors trying to build the source, so decided to use @moldach <https://github.com/moldach>'s docker image, moldach686/rufus-v1.0. Running in Singularity, I get the following error running the test script:
Singularity> sh runTest.sh
checking for samtools
/usr/local/bin/samtools
samtools found
_arg_fastqA =
_arg_fastqB =
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Final reference path being used is /tmp/RUFUS/testRun/../resources/references/small_test_human_reference_v37_decoys.fa
Final bwa reference path being used is /tmp/RUFUS/testRun/../resources/references/small_test_human_reference_v37_decoys.fa
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
proband extension is bam
you provided the proband cram file /tmp/RUFUS/testRun/Child.bam
parent file name is Mother.bam
parent file extension name is bam
You provided the control bam file /tmp/RUFUS/testRun/Mother.bam
parent file name is Father.bam
parent file extension name is bam
You provided the control bam file /tmp/RUFUS/testRun/Father.bam
value of ProbandGenerator Child.bam.generator
Value of ParentGenerators:
Mother.bam.generator
Father.bam.generator
Value of K is: 25
Value of Threads is: 40
value of ref is: /tmp/RUFUS/testRun/../resources/references/small_test_human_reference_v37_decoys.fa
value of min is:
Did not provide refHash
$_arg_min is empty
_arg_min is
MutantMinCov is
parent is Mother.bam.generator
parent is Father.bam.generator
Running jellyfish for Mother.bam.generator
/tmp/RUFUS/scripts/RunJellyForRUFUS.sh: line 34: 62244 Killed $JELLYFISH count --disk -m $K -L $L -s 8G -t $T -o $GEN.Jhash -C $GEN.fq
I also tried running this on our own data and got different errors depending on which shell I used:
with sh:
Singularity> sh ./runRufus.sh -s /scratch.global/lee04110/data/bams/Affected.bam -c /scratch.global/lee04110/data/bams/Mother.bam -c /scratch.global/lee04110/data/bams/Father.bam -c /scratch.global/lee04110/data/bams/Sister.bam -t 8 -k 25 -ref /scratch.global/lee04110/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa
checking for samtools
/usr/local/bin/samtools
samtools found
./runRufus.sh: 29: ./runRufus.sh: Bad substitution
./runRufus.sh: 50: ./runRufus.sh: Syntax error: "(" unexpected
with bash:
Singularity> bash ./runRufus.sh -s /scratch.global/lee04110/data/bams/Affected.bam -c /scratch.global/lee04110/data/bams/Mother.bam -c /scratch.global/lee04110/data/bams/Father.bam -c /scratch.global/lee04110/data/bams/Sister.bam -t 8 -k 25 -ref /scratch.global/lee04110/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa
checking for samtools
/usr/local/bin/samtools
samtools found
_arg_fastqA =
_arg_fastqB =
Reference file not built for BWA
this program requires the existence of the file ef.sa
Killing run with non-zero status
Killed
There is a properly generated .sa file in the same directory as the .fa file; I even tried renaming it "ef.sa" to no avail. Anyone have any ideas as to what is causing these errors? Is RUFUS still being actively maintained? Our lab would really like to be able to use it.
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHQJS7SSSDUG5JLMZILWD3R6HANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
Thanks very much for your quick response. I was just running the test script from Singularity on the command line and I have no idea how much memory it had available, but I can run it via a slurm job and specify enough memory. I'll also try the RDIR workaround. |
|
Since I am using the docker image, there seems to be no way work around the RDIR problem. I can't edit the script. |
|
That’s unfortunate. Since it’s a docker image, I assume that path never changes, you could ask them to change that line to just be whatever the RUFUS directory path is since it won’t ever change. Or you could do a Perl one liner at the beginning of your script that find and replaces that line with your path each run. =================================Andrew Farrell PhD Director of Research and Science Department of Human Genetics USTAR Center for Genetic Discovery Eccles Institute of Human Genetics University of Utah School of Medicine15 North 2030 East, Room 7140 Salt Lake City, UT 84112-5330 ***@***.***://marthlab.org/=================================On Oct 18, 2022, at 5:32 PM, Antares ***@***.***> wrote:
Since I am using the docker image, there seems to be no way work around the RDIR problem. I can't edit the script.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
The issue is that the script itself executes the problematic line RDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )” and so fails, regardless of anything I can set in the environment or pass to Singularity. When trying to build RUFUS from freshly checked out source, I get the following: |
|
The error in your build is due to a 3rd party program FASTP which I actually don't use anymore and should remove from the make, thanks for the reminder. For a quick fix, in the file RUFUS/externals/CMakeLists.txt comment out or remove lines 21 and 22.
include(fastp.cmake)
LIST(APPEND RUFUS_DEPENDENCIES ${FASTP_PROJECT})”
then do a fresh build and lets see if that helps.
for my first fix I was suggesting you pass a one liner to singularity when you start your program that will edit that scrip and remove that line before it runs such as
perl -ni -e 's/RDIR=\"\$\(\ cd\ \"\$\(\ dirname\ \"\$\{BASH_SOURCE\[0\]\}\"\ \)\"\ >\/dev\/null\ 2>&1\ &&\ pwd\ \)\"/RDIR=\"myRUFUSpath\"/;print’ runRufus.sh
however to be honest I haven't used singularity so have no idea if you can do that. On that note we have been given a one year grant to improve RUFUS and make it more portable so in the next year we will be working on cleaning up RUFUS and creating our own docker/singularity images.
=====================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: ***@***.***
http://marthlab.org/
=====================================
… On Oct 19, 2022, at 11:17 AM, Antares ***@***.***> wrote:
The issue is that the script itself executes the problematic line RDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )” and so fails, regardless of anything I can set in the environment or pass to Singularity.
When trying to build RUFUS from freshly checked out source, I get the following:
bwt_gen.c: In function ‘BWTIncMergeBwt’:
bwt_gen.c:953:15: warning: variable ‘bitsInWordMinusBitPerChar’ set but not used [-Wunused-but-set-variable]
unsigned int bitsInWordMinusBitPerChar;
^~~~~~~~~~~~~~~~~~~~~~~~~
[ 40%] No install step for 'bwa_project'
[ 41%] Completed 'bwa_project'
[ 41%] Built target bwa_project
Scanning dependencies of target fastp_project
[ 42%] Creating directories for 'fastp_project'
[ 43%] Performing download step (git clone) for 'fastp_project'
Cloning into 'fastp_project'...
Already on 'master'
[ 44%] No patch step for 'fastp_project'
[ 45%] No update step for 'fastp_project'
[ 45%] No configure step for 'fastp_project'
[ 46%] Performing build step for 'fastp_project'
/usr/bin/ld: cannot find -lisal
/usr/bin/ld: cannot find -ldeflate
collect2: error: ld returned 1 exit status
make[3]: *** [fastp] Error 1
make[2]: *** [externals/fastp/src/fastp_project-stamp/fastp_project-build] Error 2
make[1]: *** [externals/CMakeFiles/fastp_project.dir/all] Error 2
make: *** [all] Error 2
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHRQE2XZKZSFXEZDSWLWEAUK3ANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
RUFUS build successful! Thanks again for responding so quickly. Glad to hear that you've gotten funding for RUFUS and that a docker image is planned. |
|
The runTest script succeeded out of the box. However, when I run RUFUS with my own data, I get the same error I saw when running it from the docker image in singularity: bash $RUFUS_DIR/runRufus.sh -s /scratch.global/lee04110/data/bams/Affected.bam -c /scratch.global/lee04110/data/bams/Mother.bam -c /scratch.global/lee04110/data/bams/Father.bam -c /scratch.global/lee04110/pakistan/data/bams/Sister.bam -t 24 -k 25 -ref /scratch.global/lee04110/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa The resources directory that contains the GRCh38_full_analysis_set_plus_decoy_hla.fa fasta file does contain GRCh38_full_analysis_set_plus_decoy_hla.fa.sa as well. So I am baffled. |
|
Took me a minute but I see the problem. On your command line you put “-ref” it needs to be “-r” or “--ref” (with two dashes), “-ref” is getting interpreted as a “-r ef” so it thinks your reference file is just the string “ef”.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: ***@***.***
http://marthlab.org/
=================================
… On Oct 19, 2022, at 5:58 PM, Antares ***@***.***> wrote:
The runTest script succeeded out of the box. However, when I run RUFUS with my own data, I get the same error I saw when running it from the docker image in singularity:
bash $RUFUS_DIR/runRufus.sh -s /scratch.global/lee04110/data/bams/Affected.bam -c /scratch.global/lee04110/data/bams/Mother.bam -c /scratch.global/lee04110/data/bams/Father.bam -c /scratch.global/lee04110/pakistan/data/bams/Sister.bam -t 24 -k 25 -ref /scratch.global/lee04110/ref/GRCh38_full_analysis_set_plus_decoy_hla.fa
checking for samtools
/panfs/roc/msisoft/samtools/1.9_gcc-7.2.0_haswell/bin/samtools
samtools found
_arg_fastqA =
_arg_fastqB =
Reference file not built for BWA
this program requires the existence of the file ef.sa
Killing run with non-zero status
The resources directory that contains the GRCh38_full_analysis_set_plus_decoy_hla.fa fasta file does contain GRCh38_full_analysis_set_plus_decoy_hla.fa.sa as well. So I am baffled.
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHSQI46Z3GI66AT5H3LWECDJ5ANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
Wow, that is one embarrassing user error. Thank you. I'm thrilled to report that I've now run RUFUS successfully over some data that we are actively studying. |
|
Thats great to hear, glad it's working for you. Please let me know if you need any help understanding the output or if you get any weird results you need help with.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: ***@***.***
http://marthlab.org/
=================================
… On Oct 20, 2022, at 1:52 PM, Antares ***@***.***> wrote:
Wow, that is one embarrassing user error. Thank you. I'm thrilled to report that I've now run RUFUS successfully over some data that we are actively studying.
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHXVMZ7XR5M2KKKDTR3WEGPJNANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
Thats great to hear, glad it's working for you. Please let me know if you need any help understanding the output or if you get any weird results you need help with.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
***@***.*** ***@***.***>
http://marthlab.org/
=================================
… On Oct 20, 2022, at 1:52 PM, Antares ***@***.*** ***@***.***>> wrote:
Wow, that is one embarrassing user error. Thank you. I'm thrilled to report that I've now run RUFUS successfully over some data that we are actively studying.
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHXVMZ7XR5M2KKKDTR3WEGPJNANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
I didn't know what to make of these lines from stderr for a sample for which RUFUS successfully generated a final vcf: |
|
You can ignore those, those are warnings from one of the tools we use inside Rufus that I’m not capturing properly. Those warning are in the unaligned contigs we’re not worried about. =================================Andrew Farrell PhD Director of Research and Science Department of Human Genetics USTAR Center for Genetic Discovery Eccles Institute of Human Genetics University of Utah School of Medicine15 North 2030 East, Room 7140 Salt Lake City, UT 84112-5330 ***@***.***://marthlab.org/=================================On Oct 24, 2022, at 9:04 AM, Antares ***@***.***> wrote:
I didn't know what to make of these lines from stderr for a sample for which RUFUS successfully generated a final vcf:
Feature (HLA-A68:01:02:01:2896-3597) beyond the length of HLA-A68:01:02:01 size (3517 bp). Skipping.
Feature (HLA-DRB114:05:01:13411-13952) beyond the length of HLA-DRB114:05:01 size (13933 bp). Skipping.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Good to know, thanks. I have gotten a reproducible seg fault in RUFUSinterpret on a couple of samples, any ideas about what the problem might be? In both cases, I am able to run successfully on an affected sibling with the same controls. lee04110@ln0004 [/scratch.global/lee04110/batch] % grep 'Segmentation' *.err 151440492.err: 3173376 Segmentation fault | $RUFUSinterpret -mob ./Intermediates/$NameStub.overlap.hashcount.fastq.MOB.sam -mod $dumbFix.Jhash.histo.7.7.dist -mQ 1 -r $humanRef -hf $HashList -o ./$NameStub.overlap.hashcount.fastq.bam -m |
|
I’m sorry you're running into that, yes I have seen a segfault there that I'm trying to track down, its pretty rare so it hasn’t been a high priority but now that a use has seen it I need to move it up the todo list. RUFUS interpret has a bug that causes a segfault with alignments on some of the smaller “alt” and “random” contigs in the human genome. One quick workaround is to remove the contig causing the seg fault from analysis. I actually have this fix in the github version for “chUn”. If you look at the last line of RUFUS/scripts/OverlapShorter.sh you’ll see the last line is:
"samtools view ./$NameStub.overlap.hashcount.fastq.bam | grep -v chrUn | $RUFUSinterpret -mob ./Intermed…….”
That “grep -v chrUn” removes alignments to that decoy contig in the human genome. I would look at the end of your output and right before the segfault you should see it working a read aligned to a specific chromosome. You can just add a grep -v for the chromosome or a grep -v for the read name to remove it from the analysis. Hopefully this will fix your problem till I get that segfault fixed. Your new last line in RUFUS/scripts/OverlapShorter.sh will look something like
"samtools view ./$NameStub.overlap.hashcount.fastq.bam | grep -v chrUn | grep -v BadChromosme | $RUFUSinterpret -mob ./Intermed…….”
If the chromosome that is causing problems is one of the main chromosomes thats a bigger problem and please let me know.
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: ***@***.***
http://marthlab.org/
=================================
… On Oct 24, 2022, at 11:37 AM, Antares ***@***.***> wrote:
Good to know, thanks. I have gotten a reproducible seg fault in RUFUSinterpret on a couple of samples, any ideas about what the problem might be? In both cases, I am able to run successfully on an affected sibling with the same controls.
***@***.*** [/scratch.global/lee04110/batch] % grep 'Segmentation' *.err
151439214.err: 431062 Segmentation fault | $RUFUSinterpret -mob ./Intermediates/$NameStub.overlap.hashcount.fastq.MOB.sam -mod $dumbFix.Jhash.histo.7.7.dist -mQ 1 -r $humanRef -hf $HashList -o ./$NameStub.overlap.hashcount.fastq.bam -m $MaxAlleleSize $(echo $parentCRString) -sR Intermediates/$NameStub.overlap.asembly.hash.fastq.Ref.sample -s Intermediates/$NameStub.overlap.asembly.hash.fastq.sample -e ./Intermediates/$NameStub.ref.RepRefHash
151440492.err: 3173376 Segmentation fault | $RUFUSinterpret -mob ./Intermediates/$NameStub.overlap.hashcount.fastq.MOB.sam -mod $dumbFix.Jhash.histo.7.7.dist -mQ 1 -r $humanRef -hf $HashList -o ./$NameStub.overlap.hashcount.fastq.bam -m $MaxAlleleSize $(echo $parentCRString) -sR Intermediates/$NameStub.overlap.asembly.hash.fastq.Ref.sample -s Intermediates/$NameStub.overlap.asembly.hash.fastq.sample -e ./Intermediates/$NameStub.ref.RepRefHash
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHW3EDP74RNZK25LEYTWE3CNJANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
If you can tell from the stderr output below what contig RUFUS didn't like before it seg faulted, please enlighten me: Affected1: Affected3: Maybe it's informative that neither of these files has any chrUn reads? lee04110@ln0004 [/scratch.global/lee04110/tmp_rufus/Affected1] % samtools view ./Affected1.overlap.hashcount.fastq.bam | grep chrUn lee04110@ln0004 [/scratch.global/lee04110/tmp_rufus/Affected3] % samtools view ./Affected3.bam.generator.V2.overlap.hashcount.fastq.bam | grep chrUn Other samples that I spot-checked, which RUFUS had no problem with, did have chrUn reads, and tailing their stderr files shows output like: Feature (chrUn_JTFH01001981v1_decoy:1803-2184) beyond the length of chrUn_JTFH01001981v1_decoy size (2087 bp). Skipping. By just calling samtools view | grep chr, I see that Affected1 had reads in chromosomes chr1, chr2, chr7, chr9, chr12, chr17, chr19, chr1_KI270711v1_random, chr1_KI270711v1_random, chr1_KI270766v1_alt. Maybe RUFUS didn't like the alt? Affected3 had reads in chr1, chr2, chr4, chr16, chr19. No alt or random or chrUn. |
|
I have a different sample that the Overlap.shorter.sh script seg faults on in a different place: Affected2.bam.generator.Mutations.Mate1.fastq Affected2.bam.generator.Mutations.Mate2.fastq |
|
That actually looks like its segfaulting in a different place then I was describing before. One thing to check is that none of the input files are blank. Can you send me an “ls -l” of the directory your working in? Is there any way you can share your data with me so I can track down this bug?
=================================
Andrew Farrell PhD
Director of Research and Science
Department of Human Genetics
Eccles Institute of Human Genetics
University of Utah School of Medicine
15 North 2030 East, Room 7140
Salt Lake City, UT 84112-5330
Email: ***@***.***
http://marthlab.org/
=================================
… On Oct 25, 2022, at 6:26 PM, Antares ***@***.***> wrote:
If you can tell from the stderr output below what contig RUFUS didn't like before it seg faulted, please enlighten me:
Affected1:
[main] CMD: /home/pankrat2/public/bin/gatk4/RUFUS/scripts/..//bin/externals/bwa/src/bwa_project/bwa mem -t 24 -Y -E 0,0 -O 6,6 -d 500 -w 500 -L 0,0 /home/pankrat2/public/bin/gatk4/RUFUS/scripts/..//resources/primate_non-LTR_Retrotransposon.fasta ./Affected1.bam.generator.V2.overlap.hashcount.fastq
[main] Real time: 0.051 sec; CPU: 0.042 sec
/home/pankrat2/public/bin/gatk4/RUFUS/scripts/Overlap.shorter.sh: line 342: 3173374 Broken pipe samtools view ./$NameStub.overlap.hashcount.fastq.bam
3173375 | grep -v chrUn
3173376 Segmentation fault | $RUFUSinterpret -mob ./Intermediates/$NameStub.overlap.hashcount.fastq.MOB.sam -mod $dumbFix.Jhash.histo.7.7.dist -mQ 1 -r $humanRef -hf $HashList -o ./$NameStub.overlap.hashcount.fastq.bam -m $MaxAlleleSize $(echo $parentCRString) -sR Intermediates/$NameStub.overlap.asembly.hash.fastq.Ref.sample -s Intermediates/$NameStub.overlap.asembly.hash.fastq.sample -e ./Intermediates/$NameStub.ref.RepRefHash
Affected3:
[main] CMD: /home/pankrat2/public/bin/gatk4/RUFUS/scripts/..//bin/externals/bwa/src/bwa_project/bwa mem -t 24 -Y -E 0,0 -O 6,6 -d 500 -w 500 -L 0,0 /home/pankrat2/public/bin/gatk4/RUFUS/scripts/..//resources/primate_non-LTR_Retrotransposon.fasta ./Affected3.bam.generator.V2.overlap.hashcount.fastq
[main] Real time: 0.027 sec; CPU: 0.024 sec
/home/pankrat2/public/bin/gatk4/RUFUS/scripts/Overlap.shorter.sh: line 342: 431060 Done samtools view ./$NameStub.overlap.hashcount.fastq.bam
431061 | grep -v chrUn
431062 Segmentation fault | $RUFUSinterpret -mob ./Intermediates/$NameStub.overlap.hashcount.fastq.MOB.sam -mod $dumbFix.Jhash.histo.7.7.dist -mQ 1 -r $humanRef -hf $HashList -o ./$NameStub.overlap.hashcount.fastq.bam -m $MaxAlleleSize $(echo $parentCRString) -sR Intermediates/$NameStub.overlap.asembly.hash.fastq.Ref.sample -s Intermediates/$NameStub.overlap.asembly.hash.fastq.sample -e ./Intermediates/$NameStub.ref.RepRefHash
Maybe it's informative that neither of these files has any chrUn reads?
***@***.*** [/scratch.global/lee04110/tmp_rufus/Affected1] % samtools view ./Affected1.overlap.hashcount.fastq.bam | grep chrUn
***@***.*** [/scratch.global/lee04110/tmp_rufus/Affected1] %
***@***.*** [/scratch.global/lee04110/tmp_rufus/Affected3] % samtools view ./Affected3.bam.generator.V2.overlap.hashcount.fastq.bam | grep chrUn
***@***.*** [/scratch.global/lee04110/tmp_rufus/Affected3] %
Other samples that I spot-checked, which RUFUS had no problem with, did have chrUn reads, and tailing their stderr files shows output like:
Feature (chrUn_JTFH01001981v1_decoy:1803-2184) beyond the length of chrUn_JTFH01001981v1_decoy size (2087 bp). Skipping.
Feature (chrUn_JTFH01001981v1_decoy:1816-2187) beyond the length of chrUn_JTFH01001981v1_decoy size (2087 bp). Skipping.
Feature (chrUn_JTFH01001981v1_decoy:1816-2181) beyond the length of chrUn_JTFH01001981v1_decoy size (2087 bp). Skipping.
By just calling samtools view | grep chr, I see that Affected1 had reads in chromosomes chr1, chr2, chr7, chr9, chr12, chr17, chr19, chr1_KI270711v1_random, chr1_KI270711v1_random, chr1_KI270766v1_alt. Maybe RUFUS didn't like the alt?
Affected3 had reads in chr1, chr2, chr4, chr16, chr19. No alt or random or chrUn.
—
Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSVWHRE6657T3RFIYYSBK3WFB3FHANCNFSM4KNYDGCQ>.
You are receiving this because you were mentioned.
|
|
Working on it... |
|
The inputs to Rufus are not blank, although as mentioned there are no chrUn reads in the two input files where the segfault occurs on line 342 of the Overlap.shorter.sh script. I've attached ls -l output for the working directories. Unfortunately, it's not possible to share the data itself. I'll be happy to test fixes or run code with verbose logging enabled to help you track down the problem. |
Is there a Docker image available for RUFUS?
The text was updated successfully, but these errors were encountered: