Pindel: improved version for indels and structural variants

[KaiYe]

Originally Posted by chariko

Hi KaiYe,

I am having the same problem as jtjli (http://seqanswers.com/forums/showthr...0820#post30820). I did as follows:

1) Download all files from http://www.ebi.ac.uk/~kye/pindel/v_0.2.0/. I aligned with BWA, processed with samtools and filtered by MAPQ quality (<30).
2) ran bam2pindel.pl on one paired-end samples (aligned using BWA). My bam file is sorted and duplicates are removed but it does not have the header expected by your program, so i used the -om to force the script to run. A file for each chromosome was generated: e.g. myprefix.1.txt (chr1)
3) I downloaded your source code from sourceforge (with svn) and compiled your pindel from scratch. It seems to work.
4) I run the following comand
/home/Pindel_source_v0.2.2/pindel -f /home/hg19.fa -i /s_4_QC_sort_pind.bam_chr1.txt -o ./s4 -c chr1 empty

but whichever chromosome i try, i always get "There are no reads for this chromosome":

BreakDancer events: 0
Processing chromosome: chr10
Skipping chromosome: chr10
...

Processing chromosome: chr1
Chromosome Size: 249250621
26926 10000
Looking at chromosome chr1 bases 0 to 10000000.
BinBorder 0 10000000
There are no reads for this bin.
Looking at chromosome chr1 bases 10000000 to 20000000.
BinBorder 10000000 20000000
There are no reads for this bin.
....
Loading genome sequences and reads: 0 seconds.
Mining, Sorting and output results: 0 seconds.

What I am doing wrong? How did you solve jtjli's problem?

hi,

You should use -p for extracted reads. -i is for configuration file.

Pindel accepts two types of input:
1. extracted reads with sam2pindel or bam2pindel, using -p
2. a configure file for a list of BAMs, using -i
the format of the configure file
/path/to/bam_1/BAM_1 400 sample_1
/path/to/bam_2/BAM_2 400 sample_2
...
/path/to/bam_n/BAM_n 400 sample_n

you may also use -c chrN:start-end to specify a small region of the region to parallelize the computation.

Kai

[icg]

Originally Posted by KaiYe

Would you please inform me your email address? I have cpp code to extract reads from sam files for Pindel.

Thanks.

Hi KaiYe,

I'm trying to convert my BAM file (illumina single-end reads, aligned using Novoalign) to the pindel format, using sam2pindel but the output file is empty.

I used the following command:
./sam2pindel novo.sam Output4Pindel.txt 300 test 0

What am I doing wrong?


Thanks in advance for your reply,

Inbar

[KaiYe]

Originally Posted by icg

Hi KaiYe,

I'm trying to convert my BAM file (illumina single-end reads, aligned using Novoalign) to the pindel format, using sam2pindel but the output file is empty.

I used the following command:
./sam2pindel novo.sam Output4Pindel.txt 300 test 0

What am I doing wrong?


Thanks in advance for your reply,

Inbar

hi Inbar,

sam2pindel requires the mate information stored in each record. I guess novoalign doesn't report that.

can you provide a few lines of sam records?

Kai

[chariko]

Originally Posted by KaiYe

hi,

You should use -p for extracted reads. -i is for configuration file.

Pindel accepts two types of input:
1. extracted reads with sam2pindel or bam2pindel, using -p
2. a configure file for a list of BAMs, using -i
the format of the configure file
/path/to/bam_1/BAM_1 400 sample_1
/path/to/bam_2/BAM_2 400 sample_2
...
/path/to/bam_n/BAM_n 400 sample_n

you may also use -c chrN:start-end to specify a small region of the region to parallelize the computation.

Kai

I finally managed it to work. After following your instructions I had to change also my input files (those generated by bam2pindel) because when comparing them with the demodata, mines had only one "@" in each line instead of 2 which had the demodata. I don´t know why did that happen because I obtained those input files with bam2pindel but anyway now it worked

Thanks a lot

[icg]

Hi Kai,

Thank you for the quick reply!

Here's the first 20 lines of my sam file.

Many thanks,
Inbar

@HD VN:1.0 SO:unsorted
@PG ID:novoalign VN:V2.07.05 CL:novoalign -d NC_007530.fna.nix -f output4.fastq -r ALL -o SAM
@SQ SN:gi|50196905|ref|NC_007530.2| AS:NC_007530.fna.nix LN:5227419
@SQ SN:gi|47566322|ref|NC_007322.2| AS:NC_007530.fna.nix LN:181677
@SQ SN:gi|50163691|ref|NC_007323.3| AS:NC_007530.fna.nix LN:94830
4:1:1169:930:Y 4 * 0 0 * * 0 0 NAAACAGTGAAGTATATAACGTACATGTCNAANNNNNNNNNNNNNNGNNNNNNNNNANNNNNNNNNNNNNNNNN #+,)-23444@@8@@@@@@@C@@@C################################################# PG:Z:novoalign ZS:Z:NM
4:1:1205:937:Y 0 gi|50196905|ref|NC_007530.2| 3730834 150 1S73M * 0 0 NAAAGAAGAATTACATCGCCATCTGTAGAATGAGCATAAGCTTTCACTACCGCTTCATCTAAAGTATCGACACT #(()'3..22@@@@@@@@7@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ PG:Z:novoalign AS:i:10 UQ:i:10 NM:i:0 MD:Z:73
4:1:1231:930:Y 16 gi|50196905|ref|NC_007530.2| 899640 150 73M1S * 0 0 GAAAAGACCGAATTATCAGAATGTGTCGAATCTTCTTTTGAGAAAGTTCTTGATAACGAATGGTTTTGTATAGN 22CC@@@@@@C@@C@@@@C@C@CCC@@@@@@222@@@@@CC@@@C@C@@C@@CCC@@@@@CC2257777+000# PG:Z:novoalign AS:i:6 UQ:i:6 NM:i:0 MD:Z:73
4:1:1259:938:Y 4 * 0 0 * * 0 0 NAGCAAGGCAATGTAAAAGGCGAAAGACAAACAGCGGAAAGAGAAATTGAAATACAAAATAAATTAAGAAATAC ########################################################################## PG:Z:novoalign ZS:Z:QC
4:1:1290:941:Y 0 gi|47566322|ref|NC_007322.2| 169212 150 1S73M * 0 0 NGAAAATGCTCTTCAACTATTTGTATAGTCTATGTCACTCTTTTTTGGACTTTCCATATTGGGAGGGATGATTA #(*()00322@@@@C@C@@C@C@C@C@@@C@@@C@@@@@@@@@2222@@@C@C@@@@@C@@@@:@222:<@@@@ PG:Z:novoalign AS:i:10 UQ:i:10 NM:i:0 MD:Z:73
4:1:1307:933:Y 0 gi|50196905|ref|NC_007530.2| 4528377 150 1S73M * 0 0 NACGTAGTGGAATAGTTGAAAATTTAGATGAAGCTGATCCAGAAATTATTTTCTACACAAAAAAGCTCAGAGCA #(.((*,*))77755/0/00@@@@@@@@@@@@@:@@@@@@57055@@@@@22222@@@@@@2222@@@@@@7@@ PG:Z:novoalign AS:i:13 UQ:i:13 NM:i:0 MD:Z:73
4:1:1349:932:Y 0 gi|50196905|ref|NC_007530.2| 5190228 150 1S73M * 0 0 NGAACTATTTGAAAGATTATCTACGACTATAATTTTATAATTATTATTTAATAATTCTACACATGTATGACTAC #)*,.3103.@@@7@3<<<:@@@@@@@@@@@@@@@@@22@@@@@@@@@22@@@@@@@@@@<<<:::::::@@@@ PG:Z:novoalign AS:i:8 UQ:i:8 NM:i:0 MD:Z:73
4:1:1427:930:Y 4 * 0 0 * * 0 0 NATGTATTTGAATTATAACGTGATTCAATTTGGTTCTGGCGCAAGGAACCCAAGGGAGTTATAACTAACTCCCT ########################################################################## PG:Z:novoalign ZS:Z:QC
4:1:1455:932:Y 16 gi|50196905|ref|NC_007530.2| 4436146 150 73M1S * 0 0 CAAGACCTCCGGAATATGCTAATACAACTTTTTTCTTCTCCATTTTGCATCCCCCTAAAGAATAAATATTCATN @C@@@@C@CC@CC@@C@C@@@@@@@@@22222@@@@@@@@CC@@@CC@@@CCC@@22CC22C@C55566*(,,# PG:Z:novoalign AS:i:8 UQ:i:8 NM:i:0 MD:Z:73
4:1:1503:932:Y 0 gi|50196905|ref|NC_007530.2| 5174187 150 1S73M * 0 0 NAGAAGGAGAAACTTCAAATACAGTGAAACACCGCGATGGCCGTGTTTATGCGGAAGTAAGTGCAAAACTAACA #(*)&)*)*+<77<:58777:::::<:<<:8888885888:<:<:<<3<<:::1:@@@@@@@@@@@@@@@::<< PG:Z:novoalign AS:i:15 UQ:i:15 NM:i:0 MD:Z:73
4:1:1513:948:Y 16 gi|50196905|ref|NC_007530.2| 2854792 150 73M1S * 0 0 TGTAGAAAGTGAAAGTAAAAAAGATTCCAAAGACGCTCGTCCTTTTTCTCTATGAAATTCTTCTGCAAAATAAN C@C@C@CC@@@@@@@2222@C@@CC@@@C@C@@C@C@C@@@222C@@@C@@CCC@CCC@@@@@C71115-///# PG:Z:novoalign AS:i:6 UQ:i:6 NM:i:0 MD:Z:73
4:1:1536:944:Y 16 gi|50163691|ref|NC_007323.3| 77515 150 73M1S * 0 0 TTGCTTCAAGAAGGCGAAGAACAAATTTCTCTTTTCGATAATGTCACGCAACGAGAACAAGAAGTAAAGCTTAN @CC@@@@C@@CCC@@@C@22C@@@@@@@22@@C@@C@CC@@@@@@@@@@C@C@CC@@@@C@@C@58454,0*,# PG:Z:novoalign AS:i:7 UQ:i:7 NM:i:0 MD:Z:73
4:1:1696:942:Y 4 * 0 0 * * 0 0 NCTTATCTGCAATTGAAGGAATTAAAGTAGACAAACATTCAACTGGTGGTGTTGGTGATACAACAACATTAGTA ########################################################################## PG:Z:novoalign ZS:Z:QC
4:1:1724:952:Y 0 gi|50196905|ref|NC_007530.2| 641128 150 1S73M * 0 0 NAGATCTATTTTCGATAAAAATAACGAATGAAATTCCTACAATTGTGATGGACCAGAGAACGCCGACAAATGTA #+++-32223C22CC@@CC222222@@@@0:::::CC@@@CC@C@@@C@@CC@@CCC@CC@C@C@C@@@@@@@@ PG:Z:novoalign AS:i:6 UQ:i:6 NM:i:0 MD:Z:73
4:1:1766:932:Y 4 * 0 0 * * 0 0 NCATTAAGAAGTTTCATCATGTCCGCTGTAAACTGTTGTTCTAGTTCGTTACTTAAGACGCTTCCCTTTGAAAG ########################################################################## PG:Z:novoalign ZS:Z:QC

[KaiYe]

Originally Posted by chariko

I finally managed it to work. After following your instructions I had to change also my input files (those generated by bam2pindel) because when comparing them with the demodata, mines had only one "@" in each line instead of 2 which had the demodata. I don´t know why did that happen because I obtained those input files with bam2pindel but anyway now it worked

Thanks a lot

one @ is enough.

[DexterDuncan]

Hi Kai,

Thanks again to you and Eric_Wubbo for the new pindel and pindel2vcf. Is it still a good idea to use the filter of the bam2pinel.pl result files before using the new pindel?

Thanks,

Dex

[KaiYe]

Originally Posted by DexterDuncan

Hi Kai,

Thanks again to you and Eric_Wubbo for the new pindel and pindel2vcf. Is it still a good idea to use the filter of the bam2pinel.pl result files before using the new pindel?

Thanks,

Dex

If you use BAM files as input, you certainly don't have to use any filtering. If bam2pindel.pl is used first to extract reads, you can directly use it as input.

So you don't need to use filtering.

Kai

[DexterDuncan]

Hi Kai,

Would you briefly explain the new pindel out put for version 0.2.3 below?

Thanks,

Dex

####################################################################################################
0 D 2 NT 0 "" ChrID 20 BP 74310 74313 BP_range 74310 74316 Supports 19 18 + 9 8
- 10 10 S1 110 SUM_MS 1016 1 NumSupSamples 1 1 blood 9 8 10 10

[DexterDuncan]

Hi Kai,

Here is one SV from the new version of pindel output. Could you explain how we get the number of normal reads versus the SV reads? Also, the header now is different with the new version for each SV, and for some reason, it is not coming to me what it all means. I must admit, I need to be become more educated with SVs.

####################################################################################################
4 D 1 NT 0 "" ChrID 20 BP 34005 34007 BP_range 34005 34023 Supports 11 11 + 2 2 - 9
9 S1 30 SUM_MS 660 2 NumSupSamples 2 2 COLO-829 2 2 5 5 COLO-829-BL 0 0 4 4
CAACCAGATATGCCTCCTTACAAGAGATTCTTAAGGGAGCTCTAAACCTACAATCAAAAGAACAACACCTGCTACaAAAAAAAAAAAAAAAACATACTTATGCACATAAAGACACTATAAAGCAACTACACTATCAAGTCTACATAATAA
CTTAAGGGAGCTCTAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCAC - 34175 60 COLO-829 @@EAS188_62:6:20:111:1106/2
CAAAAAAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCACATAAAGACACTATAAAGCAACTACA + 33660 60 COLO-829 @@EAS188_62:3:40:104:1946/1
CTAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTTTGCACATAAAGACACTA - 34184 60 COLO-829 @@EAS139_60:7:37:896:889/2
AACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCAAAAAAAAACACTATAAAGCAACTACACTATCA - 34396 60 COLO-829 @@EAS139_60:5:24:381:681/1
AAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACAAACTTATGCACATAAAGACACTATA - 34196 60 COLO-829 @@EAS131_8:8:43:784:1438/2
TAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCACATAAAGACACTAT - 34388 60 COLO-829 @@EAS131_6:8:39:243:1719/1
CTCTAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCACATAAAGACAC + 33667 60 COLO-829 @@EAS25_5:1:80:1493:28/1
TCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCACATAAAGACACTATAAAGCAACTAC - 34198 60 COLO-829-BL @@USI-EAS39_8289_FC30GCV_PE:5:18:1550:123/1
CTTAAGGGAGCTCTAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCAC - 34175 60 COLO-829-BL @@HWI-EAS300_8282_FC30BVC_PE:1:15:777:1187/1
TAAGGGAGCTCTAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCACAT - 34164 60 COLO-829-BL @@HWI-EAS255_8291_FC30GRN_PE:2:73:533:1356/2
GAGCTCTAAACCTACAATCAAAAGAACAACACCTGCTAC AAAAAAAAAAAAAAAACATACTTATGCACATAAAGA - 34192 60 COLO-829-BL @@HWI-EAS138_4_FC30GP8:4:54:1227:1320/2


Thanks for all of your assistance,

Dex

[glede]

I think you can refer to the manual at https://trac.nbic.nl/pindel/wiki/UserManual

[mig174]

I ran into some problems while trying to use pindel:
1. is there a 32bit-version?
2. while trying to compile version 0.2.4 there was an error with TIME_MINE_E, which is only set once to time(null) and never used in the rest of the code; as i did not see a possibility to fix that without writing around in your code, i decided to download version 0.2.0
3. after downloading version 0.2.0 and trying to run it with the parameters described on https://trac.nbic.nl/pindel/wiki/UserManual i only got the following usage information:

Welcome to Pindel, developed by Kai Ye, k.ye@lumc.nl

7 parameters are required here:
1. Input: the reference genome sequences in fasta format;
2. Input: the unmapped reads in a modified fastq format;
Better to use bam2pindel.pl to convert BAM files to Pindel input.
If the perl script fails for some reasons, please use the provided
sam2pindel.cpp to extract reads from sam files.
Compile cpp file first: g++ sam2pindel.cpp -o sam2pindel -O3
3. Output folder
4. BreakDancer result:
ChrA LocA stringA ChrB LocB stringB others
If you don't have BreakDancer result, please provide an empty file here.
5. Maximum event size index. 5 is recommended
2: 128
3: 512
4: 2,048
5: 8,092
6: 32,368
7: 129,472
8: 517,888
9: 2,071,552
10: 8,286,208
11: 33,144,832
12: 132,579,328
6. Number of threads
7. Which chr/fragment
Pindel will process reads for one chr each time
ChrName must be the same as in reference sequence and in read file

so it seems as if the information on the homepage is no longer accurate

finally i got it to run and it seems to work perfectly (thanks for that :-) ) but the way of getting there was a little painful.

[KaiYe]

Originally Posted by mig174

I ran into some problems while trying to use pindel:
1. is there a 32bit-version?
2. while trying to compile version 0.2.4 there was an error with TIME_MINE_E, which is only set once to time(null) and never used in the rest of the code; as i did not see a possibility to fix that without writing around in your code, i decided to download version 0.2.0
3. after downloading version 0.2.0 and trying to run it with the parameters described on https://trac.nbic.nl/pindel/wiki/UserManual i only got the following usage information:

so it seems as if the information on the homepage is no longer accurate

finally i got it to run and it seems to work perfectly (thanks for that :-) ) but the way of getting there was a little painful.

1. Can you indicate which linux distribution you are using? I would install it on my PC to test it.
2. There are many nice features and improvement in new version. It is strongly advised to use the new version. And current stable version is 0.2.4h. I would like to you help you compiling the latest version. Please communicate with me by email, k.ye@lumc.nl
3. Current user manual is for the versions after 0.2.3, which has different but a more user friendly interface.
4. We support both BWA and mosaik BAMs now.

Please contact me by email (k.ye@lumc.nl) or phone (+31 71 526 9745). We will find a solution for you to run the latest version of Pindel.

Cheers,

Kai

[fyusufi]

Hi Kai,

I was wondering if there were any special parameters required while running BWA, or are the default values ok?

Also, if I have sequence libraries with different insert sizes should I map them into separate bam files?

Thanks!

[KaiYe]

Originally Posted by fyusufi

Hi Kai,

I was wondering if there were any special parameters required while running BWA, or are the default values ok?

Also, if I have sequence libraries with different insert sizes should I map them into separate bam files?

Thanks!

Default parameters of BWA is fine.

If you have multiple insert sizes, it is better to map them separately. It is possible to put them in one BAM if the insert sizes don't differ very much, say 400 vs 500. Pindel will internally double the number so that slightly underestimation is fine.

Kai

[fyusufi]

Thanks for the quick reply!

[libiyagirl]

Hey Kai,

I am just wondering about the power of Pindel to detect long insertions. When I was running Pindel on a single chromosome, I didn't find any large insertions. But actually there are some duplications in that chromosome identified by breakdancer. Did I miss something? The "zero" LI looks very strange to me.

thanks a lot,

[KaiYe]

Originally Posted by libiyagirl

Hey Kai,

I am just wondering about the power of Pindel to detect long insertions. When I was running Pindel on a single chromosome, I didn't find any large insertions. But actually there are some duplications in that chromosome identified by breakdancer. Did I miss something? The "zero" LI looks very strange to me.

thanks a lot,

Please check whether you switch on long insertion option. In some versions, this function is turned off by default.

Kai

[ralonso]

Hello,

I am about to use pindel and I am wondering some questions.
I have a file mapped with bwa and illumnia reads.
As I see in the pindel webpage https://trac.nbic.nl/pindel/wiki/UserManual I should use "unmappable read". I have several bams, so I don't know which to use, those are:
1. sample.bam: mapped and unmmapped reads,it is not sorted
2. sample_mapped.bam: just mapped reads, it is not sorted, it has not unmapped
3. sample_mapped_singlehit_sorted.bam: just mapped reads, it is sorted, it has only single hit and it is not unmapped

should I use one of this files? or maybe another with the unmappable reads?

thank you very much!

[KaiYe]

Originally Posted by ralonso

Hello,

I am about to use pindel and I am wondering some questions.
I have a file mapped with bwa and illumnia reads.
As I see in the pindel webpage https://trac.nbic.nl/pindel/wiki/UserManual I should use "unmappable read". I have several bams, so I don't know which to use, those are:
1. sample.bam: mapped and unmmapped reads,it is not sorted
2. sample_mapped.bam: just mapped reads, it is not sorted, it has not unmapped
3. sample_mapped_singlehit_sorted.bam: just mapped reads, it is sorted, it has only single hit and it is not unmapped

should I use one of this files? or maybe another with the unmappable reads?

thank you very much!

sorted sample.bam will be fine. Pindel reads bam files directly and get useful reads by itself.

You need to use config file to tell Pindel where are your files, insert size and sample names.

Kai