SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Samtools sort problem twotwo Bioinformatics 2 07-05-2013 09:31 AM
About samtools sort Richard.Y Genomic Resequencing 1 07-04-2013 07:34 AM
samtools sort hanshart Bioinformatics 4 07-01-2013 07:45 AM
***samtools*** sort wrong??? shuoguo Bioinformatics 2 09-21-2012 05:47 PM
samtools sort EBER Bioinformatics 1 06-08-2012 05:15 PM

Reply
 
Thread Tools
Old 07-11-2014, 12:51 AM   #1
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Post I am mad because of samtools sort command

I got .sam files from Bowtie2.
now I want to merge these two files.First ,I run:
samtools view -bSh ERR1.sam >ERR1.bam
samtools view -bSh ERR2.sam >ERR2.bam

and,I got the bam file.(they should have the head)
However,I run the next:
samtools sort ERR1.bam ERR1.sorted.bam (here,I got the sorted file,lucky)
samtools sort ERR2.bam ERR2.sorted.bam
about the ERR2.bam, I didn't get the sorted file, this was the output:

[bam_header_read] invalid BAM binary header (this is not a BAM file).
[bam_sort_core] truncated file. Continue anyway.
Segmentation fault (core dumped)

why?Just because the ERR2.sam is too big(about 66G)?
Lv Ray is offline   Reply With Quote
Old 07-11-2014, 01:45 AM   #2
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Default

supplement:
I run command: samtools view ERR2.bam |less -S
and I got this:
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[main_samview] fail to read the header from "ERR173170_paired.bam".
Lv Ray is offline   Reply With Quote
Old 07-11-2014, 03:04 AM   #3
biocomputer
Member
 
Location: Canada

Join Date: Dec 2013
Posts: 62
Default

If the file is too big for sorting you could split the .sam file on chromosome, sort each, recombine, then convert to .bam.
biocomputer is offline   Reply With Quote
Old 07-11-2014, 03:40 AM   #4
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Question

Quote:
Originally Posted by biocomputer View Post
If the file is too big for sorting you could split the .sam file on chromosome, sort each, recombine, then convert to .bam.
You mean that i got the fault in producing the sam file?but my sam file is okay!
Lv Ray is offline   Reply With Quote
Old 07-11-2014, 04:13 AM   #5
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Default

like your mean, maybe I need to split my big sam file
Lv Ray is offline   Reply With Quote
Old 07-11-2014, 06:28 AM   #6
dpryan
Devon Ryan
 
Location: Freiburg, Germany

Join Date: Jul 2011
Posts: 3,480
Default

66 gigs isn't too big to sort, the original BAM file was corrupt, likely due to running out of space or a hardware problem. Make sure you have enough space and then remake the BAM file.
dpryan is offline   Reply With Quote
Old 07-11-2014, 07:25 AM   #7
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Default

The space is enough, how about the memory?
Lv Ray is offline   Reply With Quote
Old 07-11-2014, 07:30 AM   #8
dpryan
Devon Ryan
 
Location: Freiburg, Germany

Join Date: Jul 2011
Posts: 3,480
Default

The whole SAM file isn't loaded into memory, it's processed line by line (and compressed in blocks).
dpryan is offline   Reply With Quote
Old 07-11-2014, 07:30 AM   #9
dpryan
Devon Ryan
 
Location: Freiburg, Germany

Join Date: Jul 2011
Posts: 3,480
Default

A transient hardware error is the most likely cause of this sort of thing.
dpryan is offline   Reply With Quote
Old 07-11-2014, 09:08 AM   #10
arundurvasula
Member
 
Location: California

Join Date: Jun 2014
Posts: 16
Default

I recently got an error like that because I switched my reads with my reference sequence while mapping. I.e. my alignment was of my reference to my reads. Maybe that's your problem?

Here's a (correct) bash function that I used to map reads to a reference and only grab the mapped reads from the sam. Hopefully this can help guide you:
Code:
map () {
	bwa index -a bwtsw $refseq
	bwa bwasw $refseq ../temp/$1/sampled_reads.fasta > ../temp/$1/alignment.sam
	samtools view -bS -F 4 ../temp/$1/alignment.sam > ../temp/$1/mapped.alignment.bam
	samtools sort ../temp/$1/mapped.alignment.bam ../results/$1/mapped.sorted.alignment
	samtools index ../results/$1/mapped.sorted.alignment.bam
}
arundurvasula is offline   Reply With Quote
Old 07-11-2014, 06:31 PM   #11
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Default

Thank you a lot for sparing your beautiful bash.
However ,it seems that your bash is suitable for unpaired alignment. I thought that because the command samtools view -bS -F 4 ../temp/$1/alignment.sam > ../temp/$1/mapped.alignment.bam ,you discard the unmapped reads. But how to set the parameter -F In paired alignment reads (.sam) ?
Thanks all .
Lv Ray is offline   Reply With Quote
Old 07-12-2014, 02:01 AM   #12
dpryan
Devon Ryan
 
Location: Freiburg, Germany

Join Date: Jul 2011
Posts: 3,480
Default

-F 4 will remove unmapped reads in either case. If you want to remove those reads with an unmapped mate then just filter according to that bit in the flag.
dpryan is offline   Reply With Quote
Old 07-14-2014, 05:49 PM   #13
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Default About the parameter -F of samtools view

Quote:
Originally Posted by dpryan View Post
-F 4 will remove unmapped reads in either case. If you want to remove those reads with an unmapped mate then just filter according to that bit in the flag.
Hi, dpryan. Thank you for your answer. Now I have the similar quetions about -F, . I appreciate and hope you can help me.
1) Should I remove the unmapped reads (but its mates mapped) or the unmapped mates(but its reads mapped)
2) about the paired reads, if I remove all them above ,should I use -F 12? However , it seems that there's no the value of 12. How about the 77 or 141.
Lv Ray is offline   Reply With Quote
Old 07-15-2014, 03:54 AM   #14
dpryan
Devon Ryan
 
Location: Freiburg, Germany

Join Date: Jul 2011
Posts: 3,480
Default

1) It depends on what you want to do with the results.
2) -F 12 is correct. There don't have to be any flags with that value since this is a bit comparison.
dpryan is offline   Reply With Quote
Old 07-15-2014, 04:30 AM   #15
Lv Ray
Member
 
Location: GZ,China

Join Date: Jun 2014
Posts: 42
Default

Thank you, dpryan.
Today I tested it ,and the result is Consistent with your answer!
Thanks, again.
Lv Ray is offline   Reply With Quote
Reply

Tags
samtools sort

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 06:51 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO