samtools: parse error in SAM to BAM conversion

chrisW · 06-11-2010, 03:05 PM

Greetings, I am a novice user with little experience running command line software. I am enjoying learning, though error messages leave me at a loss.

Background: I used bwa to create my SAM file. When I attempt to use the "view" option to convert to BAM I receive the following:

[samopen] SAM header is present: 100 sequences.
Parse error at line 18750817: unmatched CIGAR operation
Aborted

I am unclear as to what the problem is. Any guidance?

-Chris

nilshomer · 06-12-2010, 08:02 AM

Quote:

Originally Posted by chrisW

Greetings, I am a novice user with little experience running command line software. I am enjoying learning, though error messages leave me at a loss.

Background: I used bwa to create my SAM file. When I attempt to use the "view" option to convert to BAM I receive the following:

[samopen] SAM header is present: 100 sequences.
Parse error at line 18750817: unmatched CIGAR operation
Aborted

I am unclear as to what the problem is. Any guidance?

-Chris

It may be a bug in the program that produced the BAM; you may be able to post on that program's help mailing list. Optionally, you could print out line "18750817" to show us the offending line.

chrisW · 06-14-2010, 12:06 PM

Thank you for getting back to me. I don't know how to get text from the terminal and paste it into these forum windows. I did, however, look at the offending line. It is missing the actual nucleotide and associated Illumina information. Is there an easy way to explain how I can get terminal screen printout into a form I can "cut/paste"? I could then show you exactly what I'm seeing.

Based on your comments, it appears that bwa perhaps introduced an error in creating the SAM file ...or there may be something off in the datafiles I'm feeding into bwa?

Again, I appreciate your willingness to help. There is a steep learning curve for me as I've never had any Unix training. I just learned the "sed" function to look at the offending line and see what was wrong.
-Chris

lh3 · 06-14-2010, 12:37 PM

is it the last line in output? i guess it is truncated.

chrisW · 06-14-2010, 01:36 PM

No, it's not the last line in output. I'm not sure of the line's exact position in the SAM file, but I printed out the lines prior and after the offending line and they are intact with a nucleotide string followed by what I understand to be Illumina information (perhaps pertaining to quality of the nucleotide call?)

I would show you what I'm seeing but I don't know how to get output from my terminal window into a form that I can post, other than typing verbatim what I'm seeing into these chat windows. There must be an easier way.

At this point I don't know if this offending line is the only one in my SAM file or if there are others. It just appears to be the first and thus aborts the SAM --> BAM conversion.

Thanks for you help.

chrisW · 06-14-2010, 03:56 PM

Figured out my terminal to chat cut/paste. So, here are three lines from my .sam file. The middle line is the offending line that brings up the "unmatched CIGAR operation".

output/

chriswall@ubuntu:/host/Users/chris.wall/Desktop/Mastigo-genomics/bwa_cw$ sed '18750816,18750818!d' WC.sam
ILLUMINA-2F52BD:6:50:1169:1446#0 163 NODE_37776_length_48465_cov_160.191483\par 35442 29 59M = 35600 217 TTTTATCGGTGTGTATCGGTGTGTATCGGTGGGCGAGTTTTCAACAAGATTATGGGGCA gggfggggfdae[_aee_eeZ_^]_`aaeS]\FY\`dWZ]_^`_`WcSZXg`dgfcbYV XT:A:U NM:i:2 SM:i:0 AM:i:0 X0:i:1 X1:i:2 XM:i:2 XO:i:0 XG:i:0 MD:Z:7T27G23

ILLUMINA-2F52BD:6:50:1169:1947#0 83 NODE_51983_length_172857_cov_173.044144\par 119160 60 5 XT:A:U NM:i:1 SM:i:37 AM:i:37 X0:i:1 X1:i:0 XM:i:1 XO:i:0 XG:i:0 MD:Z:39C19

ILLUMINA-2F52BD:6:50:1178:417#0 83 NODE_43256_length_74978_cov_171.684830\par 68201 60 59M = 67821 -439 CAGAGAAAAGACAGGTGATTATCAGACAACCTACGGATTGATAGGAATTTTGAACCGCA bgdgbgfgf^dddddgdfgffggfbaggZefeae\ggggbfgggegfggggcg`feffe XT:A:U NM:i:0 SM:i:37 AM:i:37 X0:i:1 X1:i:0 XM:i:0 XO:i:0 XG:i:0 MD:Z:59

/output

Can this offending line and perhaps any subsequent offending lines be deleted from the .sam file and not introduce a problem?

nilshomer · 06-14-2010, 04:02 PM

The sequence and qualities are also missing along with the CIGAR. What program generated the SAM file?

chrisW · 06-14-2010, 04:10 PM

bwa 0.5.5 package from Ubuntu

nilshomer · 06-14-2010, 04:39 PM

Quote:

Originally Posted by chrisW

bwa 0.5.5 package from Ubuntu

Try the latest source code from the BWA SVN repository.

lh3 · 06-15-2010, 05:30 AM

actually 0.5.5 is more stable than the latest release. 1KG has been using it to map >20TB of data and it never fails.

nilshomer · 06-15-2010, 09:01 AM

Any suggestions for the missing CIGAR?

lh3 · 06-15-2010, 09:20 AM

I guess this is a disk error or input error. It is quite unlikely to happen if you read the source code on printing alignments. It has never happened before.

DNASpeaks · 01-14-2013, 06:16 PM

Quote:

Originally Posted by lh3

I guess this is a disk error or input error. It is quite unlikely to happen if you read the source code on printing alignments. It has never happened before.

Hello Everyone,
I am having trouble converting the sam files to bam using samtools. I used this cmd to convert my sam to bam -

samtools view -bT hg19.fa s_chip2.sam > s_chip2.bam
I got this error.
Parse error at line 7707082: sequence and quality are inconsistent
Aborted

I ran ValidateSamFile.jar, a picard tool and got the following error, hundreds of them -

WARNING: Read name HWI-ST798R:82: D18MUACXX:2:1101:2025:1987, A record is missing a read group
ERROR: Record 5, Read name HWI-ST798R:82: D18MUACXX:2:1101:8625:1992, Empty sequence dictionary.

I am not sure how to fix this, any help is appreciated.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
sam to bam conversion error, no @SQ lines in the header, missing header?	efoss	Bioinformatics	17	12-03-2015 04:28 AM
error with sam output ->Parse error at line xxxxx: missing colon in auxiliary data	manore	Bioinformatics	11	11-25-2013 01:50 PM
BWA sam and Samtools sam->bam conversion problem	maasha	Bioinformatics	6	06-05-2013 07:39 AM
Tophat v1.1.4 potential error with sam to bam conversion?	jb2	Bioinformatics	6	11-17-2011 12:52 AM
Samtools....SAM to BAM...warning or error!! Can I ignore?	Siva	Bioinformatics	12	08-20-2010 06:49 AM

06-11-2010, 03:05 PM	#1
chrisW Junior Member Location: Montana Join Date: Jun 2010 Posts: 7	samtools: parse error in SAM to BAM conversion Greetings, I am a novice user with little experience running command line software. I am enjoying learning, though error messages leave me at a loss. Background: I used bwa to create my SAM file. When I attempt to use the "view" option to convert to BAM I receive the following: [samopen] SAM header is present: 100 sequences. Parse error at line 18750817: unmatched CIGAR operation Aborted I am unclear as to what the problem is. Any guidance? -Chris

06-14-2010, 12:06 PM	#3
chrisW Junior Member Location: Montana Join Date: Jun 2010 Posts: 7	Thank you for getting back to me. I don't know how to get text from the terminal and paste it into these forum windows. I did, however, look at the offending line. It is missing the actual nucleotide and associated Illumina information. Is there an easy way to explain how I can get terminal screen printout into a form I can "cut/paste"? I could then show you exactly what I'm seeing. Based on your comments, it appears that bwa perhaps introduced an error in creating the SAM file ...or there may be something off in the datafiles I'm feeding into bwa? Again, I appreciate your willingness to help. There is a steep learning curve for me as I've never had any Unix training. I just learned the "sed" function to look at the offending line and see what was wrong. -Chris

06-14-2010, 12:37 PM	#4
lh3 Senior Member Location: Boston Join Date: Feb 2008 Posts: 693	is it the last line in output? i guess it is truncated.

06-14-2010, 01:36 PM	#5
chrisW Junior Member Location: Montana Join Date: Jun 2010 Posts: 7	No, it's not the last line in output. I'm not sure of the line's exact position in the SAM file, but I printed out the lines prior and after the offending line and they are intact with a nucleotide string followed by what I understand to be Illumina information (perhaps pertaining to quality of the nucleotide call?) I would show you what I'm seeing but I don't know how to get output from my terminal window into a form that I can post, other than typing verbatim what I'm seeing into these chat windows. There must be an easier way. At this point I don't know if this offending line is the only one in my SAM file or if there are others. It just appears to be the first and thus aborts the SAM --> BAM conversion. Thanks for you help.

06-14-2010, 03:56 PM	#6
chrisW Junior Member Location: Montana Join Date: Jun 2010 Posts: 7	Figured out my terminal to chat cut/paste. So, here are three lines from my .sam file. The middle line is the offending line that brings up the "unmatched CIGAR operation". output/ chriswall@ubuntu:/host/Users/chris.wall/Desktop/Mastigo-genomics/bwa_cw$ sed '18750816,18750818!d' WC.sam ILLUMINA-2F52BD:6:50:1169:1446#0 163 NODE_37776_length_48465_cov_160.191483\par 35442 29 59M = 35600 217 TTTTATCGGTGTGTATCGGTGTGTATCGGTGGGCGAGTTTTCAACAAGATTATGGGGCA gggfggggfdae[_aee_eeZ_^]_`aaeS]\FY\`dWZ]_^`_`WcSZXg`dgfcbYV XT:A:U NM:i:2 SM:i:0 AM:i:0 X0:i:1 X1:i:2 XM:i:2 XO:i:0 XG:i:0 MD:Z:7T27G23 ILLUMINA-2F52BD:6:50:1169:1947#0 83 NODE_51983_length_172857_cov_173.044144\par 119160 60 5 XT:A:U NM:i:1 SM:i:37 AM:i:37 X0:i:1 X1:i:0 XM:i:1 XO:i:0 XG:i:0 MD:Z:39C19 ILLUMINA-2F52BD:6:50:1178:417#0 83 NODE_43256_length_74978_cov_171.684830\par 68201 60 59M = 67821 -439 CAGAGAAAAGACAGGTGATTATCAGACAACCTACGGATTGATAGGAATTTTGAACCGCA bgdgbgfgf^dddddgdfgffggfbaggZefeae\ggggbfgggegfggggcg`feffe XT:A:U NM:i:0 SM:i:37 AM:i:37 X0:i:1 X1:i:0 XM:i:0 XO:i:0 XG:i:0 MD:Z:59 /output Can this offending line and perhaps any subsequent offending lines be deleted from the .sam file and not introduce a problem?

06-14-2010, 04:02 PM	#7
nilshomer Nils Homer Location: Boston, MA, USA Join Date: Nov 2008 Posts: 1,285	The sequence and qualities are also missing along with the CIGAR. What program generated the SAM file?

06-14-2010, 04:10 PM	#8
chrisW Junior Member Location: Montana Join Date: Jun 2010 Posts: 7	bwa 0.5.5 package from Ubuntu

06-15-2010, 05:30 AM	#10
lh3 Senior Member Location: Boston Join Date: Feb 2008 Posts: 693	actually 0.5.5 is more stable than the latest release. 1KG has been using it to map >20TB of data and it never fails.

06-15-2010, 09:01 AM	#11
nilshomer Nils Homer Location: Boston, MA, USA Join Date: Nov 2008 Posts: 1,285	Any suggestions for the missing CIGAR?

06-15-2010, 09:20 AM	#12
lh3 Senior Member Location: Boston Join Date: Feb 2008 Posts: 693	I guess this is a disk error or input error. It is quite unlikely to happen if you read the source code on printing alignments. It has never happened before.