Snakefile input

[s-canchi]

• Can we use $input or similar in the command?
• Input check to see if the input file is available?
• Does snakemake look for the input file with "input:" only if it was explicitly define as "output:" in a previous rule? or does it also look into the actual lines with commands like "wget something", "gunzip something"?
• Related question - can you put an entire folder in {input} if you have many similar files?
• Could you insert a regular expression here for extensions in defining inputs or outputs?

[marisalim]

Re: Can we use $input or similar in the command?

Yes! Here is an example of how that would work --

Our rule:

rule uncompress_genome:
    input: "ecoli-rel606.fa.gz"
    output: "ecoli-rel606.fa"
    shell:
        "gunzip ecoli-rel606.fa.gz"

Modified to use {input}:

rule uncompress_genome:
    input: "ecoli-rel606.fa.gz"
    output: "ecoli-rel606.fa"
    shell:
        "gunzip {input}"

[marisalim]

Re: Could you insert a regular expression here for extensions in defining inputs or outputs?

Yes!

Alternatively, you can use pattern expansion and matching functions:

• see glob_wildcards
• here is an example of pattern expansion and matching using the expand() function from snakemake -

Our rule:

rule index_genome_bwa:
    input: "ecoli-rel606.fa"
    output: "ecoli-rel606.fa.pac",
        "ecoli-rel606.fa.ann",
        "ecoli-rel606.fa.amb",
        "ecoli-rel606.fa.bwt",
        "ecoli-rel606.fa.sa"
    shell:
        "bwa index {input}"

Modified to use the expand() function from snakemake. All the files begin with ecoli-rel606.fa. but there are 5 different file extensions, which we specify with ext extensions using python list format:

rule index_genome_bwa:
    input: "ecoli-rel606.fa"
    output:
        expand("ecoli-rel606.fa.{ext}", ext=['sa', 'amb', 'ann', 'pac', 'bwt'])
    shell:
        "bwa index {input}"

[ctb]

although note that this is not using regular expressions, but rather pattern matching and programmatic pattern expansion.

[marisalim]

Re: Related question - can you put an entire folder in {input} if you have many similar files?

This can get complicated to keep track of - i.e., you need to make sure that the directory is generated in the snakemake workflow so it can be linked properly across rules and be re-created if the workflow is rerun. An alternative is to look at using wildcards to define multiple inputs with similar file name patterns. On a slightly related note, here is snakemake documentation on using directories as outputs.

[ctb]

The guiding principle here is that snakemake must "own" whatever the output is, and moreover the directory name must be matched to any inputs that want to use it.

So, if you specify a directory as an output for a rule, then all content in that entire directory must be created by the rule.

Also, you cannot have a directory as an output in one rule, and then have a file within that directory be an input to another rule. (what the shell command does can of course be different; this is just talking about the workflow logic of input/output and rules.)

[marisalim]

Re:

• Input check to see if the input file is available?
• Does snakemake look for the input file with "input:" only if it was explicitly define as "output:" in a previous rule? or does it also look into the actual lines with commands like "wget something", "gunzip something"?

Snakemake does check whether an input already exists as long as the input/output blocks are specified in the rule. You should see this message from snakemake if the rule's outputs already exist when you try to run it again:

Building DAG of jobs...
Nothing to be done.

Snakemake does not interpret input/output names in the shell block, and this is why it's important to define files in the input and output blocks!