Getting SRA data into AWS EC2 instance

[s-canchi]

How can I get data from SRA repository into AWS instance ?

[s-canchi]

This mini tutorial will walk you through the steps of obtaining SRA data in the public domain into an EC2 instance. We are choosing this dataset to demonstrate. You can extend this to download any data if you know the Run Accession number.

1. Create an EC2 t2.micro instance in N.California with a Ubuntu 20.04 Focal AMI (steps from the AWS lesson). Note that although the data is hosted in S3 buckets hosted in N.Virginia region, these data are part of the AWS Registry of Open data and hence the intra regional transfer fee has been waived.
2. Install unzip as it is needed for AWS CLI.
   • sudo apt install unzip
3. Install the AWS CLI:
   • curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
   • unzip awscliv2.zip
   • sudo ./aws/install
   • check installation with aws --version
4. Look at the available files in the public domain of SRA
   • aws s3 ls s3://sra-pub-run-odp/sra --no-sign-request
5. Check the existence of the file of interest using the Run Accession listed in this table
   • aws s3 ls s3://sra-pub-run-odp/sra/SRR4423266 --no-sign-request --recursive --summarize --human-readable
   • The above command will list one file of size 617.5 MiB
6. Create a data folder and change into it
   • mkdir sra_data
   • cd sra_data
7. Copy the file into the data folder
   • aws s3 cp s3://sra-pub-run-odp/sra/SRR4423266/ . --no-sign-request --recursive
8. You can view the contents of the folder using ls -lh. Since it is a binary file the contents are not viewable.

Note that this method only allows for download of a single file at a time. You can scale using a for loop to go over the list of all interested accession numbers.