Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • How long does it take to map a human genome?

    Hi,
    I've sent off several human DNA extractions to companies like Novogene or Macrogene and they returned 15x - 30x genome data in FastQ and / or BAM files aligned to hg19. (paired end 2 x 150 bases)

    Now I wanted to try and map the FastQ.gz files to hg38 by myself. I have a 16 core (32 thread) 2 GHz machine with 128 GByte RAM and a ~700 GByte fast SSD. Before I'll get fancy and try BBmap I wanted to test an old school approach with bwa mem to see how it works. I compiled the newest bwa from github. Indexing the hg38 was fast and easy. However now I have a bad feeling about the bwa mem mapping because it takes already 15 hours and I don't see any progress. Here is what I have:

    thomas@streymoy:/mnt/raid0ssd/align/2987$ ~/src/bwa/bwa mem -t 30 -M /mnt/raid0ssd/refseq/hg38/hg38.fa /mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_1.fq.gz /mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_2.fq.gz > S_2987_D16084169_HWM7HCCXX_L1_pe.sam
    [M::bwa_idx_load_from_disk] read 0 ALT contigs
    [M:rocess] read 2000000 sequences (300000000 bp)...
    [M:rocess] read 2000000 sequences (300000000 bp)...

    The paired end FastQ files are about 8 GBytes large and contain 85,069,589 reads each.
    Is it right that only 2 million reads are being processed until now?

    I wonder if my machine is under-powered or if I'm doing something completely wrong.
    htop shows that in fact 30 threads are running at close to 100%.

    Thanks for your thoughts.

  • #2
    That sounds way slower than I would expect. Some reads can take much longer to map than others, if the error rate is high, for example, but even so, bwa-mem does local alignments and soft-clips low-quality portions, so it still tends to be fairly fast. I'm guessing that there's a bug in that version, or else you have a very peculiar dataset.

    You can estimate the speed of mapping with BBMap using just the first, say, 100,000 reads, like this:
    bbmap.sh in1=r1.fastq in2=r2.fastq ref=hg38.fasta out=mapped.sam reads=100000

    You may want to look at some metrics on the sam file generated thus far to get an idea of whether the reads are the culprit. For example, a very high mismatch or indel rate *might* explain the low speed.

    Comment


    • #3
      Brian,
      Thanks for your comment. I think I have something wrong with the bwa mem parameters.
      I know the mismatch and indel rate from the hg19 bam file, which shouldn't be that much different to hg38 I guess.

      SNV Statistics (hg19)
      Sample exonic intronic UTR3 UTR5 intergenic ncRNA_exonic ncRNA_intronic upstream downstream splicing ncRNA_splicing
      S_2987 28145 1464007 29891 6970 2402149 15160 232138 28093 28216 3197 432

      So this should be 4.2 million SNVs total, which sounds reasonable with an average 3.6 million SNVs per human.
      Note it's an African sample and the hg19 reference is essentially European.

      I don't think the quality of the dataset is too bad according to FastaQC. It may contain bacterial DNA though, since it was a spit sample.

      I think I'll cancel the bwa mem process and try with BBMap. I guess I need to re-index the reference when switching from bwa to BBMap, right?
      What would be the rough estimated mapping time with my setup? minutes? hours? days? or weeks?

      Comment


      • #4
        The command I gave will do the indexing prior to mapping, which takes a few minutes; BBMap can't use bwa indexes. I just ran mapping of 2 million synthetic paired 2x150bp reads to the human genome on a 20-core machine, which took 214 seconds; that's roughly the time frame you should expect for high-quality real data. Low-quality data will be slower, but certainly not days or weeks.

        Comment


        • #5
          Thanks again Brian.

          I tried 100000 (paired) reads with BBMap as you suggested. With previously separate indexing of the reference the total mapping time was 677.5 seconds (~11 minutes).

          So 85,069,589 (paired) reads should take 576,346 seconds = 9606 minutes = 160 hours = 6.7 days = almost one week

          Wow! that looks like my machine is a little smaller than I expected in the whole genomic category...
          Will need to see if I get some more CPUs...
          That's probably why bwa mem didn't show any progress.

          Comment


          • #6
            Can you print the stderr output from BBMap? That includes some useful data about the error rates, which might be helpful in diagnosing this issue. I'm guessing there's something odd about the reads. Also, from the mapped file, you can produce some histograms like this:

            reformat.sh in=mapped.sam mhist=mhist.txt qhist=qhist.txt qahist=qahist.txt aqhist=aqhist.txt bhist=bhist.txt

            Running FastQC might also be helpful.

            Comment


            • #7
              Here you go:
              Code:
              thomas@streymoy:/mnt/raid0ssd/align/2987$ /home/thomas/src/bbmap/bbmap.sh in1=/mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_1.fq in2=/mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_2.fq out=S_2987_D16084169_HWM7HCCXX_L1.bbmap.sam reads=100000
              java -Djava.library.path=/home/thomas/src/bbmap/jni/ -ea -Xmx70125m -cp /home/thomas/src/bbmap/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 in1=/mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_1.fq in2=/mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_2.fq out=S_2987_D16084169_HWM7HCCXX_L1.bbmap.sam reads=100000
              Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, in1=/mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_1.fq, in2=/mnt/raid0ssd/fastq/2987/S_2987_D16084169_HWM7HCCXX_L1_2.fq, out=S_2987_D16084169_HWM7HCCXX_L1.bbmap.sam, reads=100000]
              
              BBMap version 36.59
              Retaining first best site only for ambiguous mappings.
              Max reads: 100000
              Set genome to 1
              
              Loaded Reference:	5.117 seconds.
              Loading index for chunk 1-8, build 1
              Generated Index:	10.184 seconds.
              Analyzed Index:   	11.591 seconds.
              Started output stream:	0.031 seconds.
              Cleared Memory:    	2.565 seconds.
              Processing reads in paired-ended mode.
              Started read stream.
              Started 32 mapping threads.
              Detecting finished threads: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
              
                 ------------------   Results   ------------------   
              
              Genome:                	1
              Key Length:            	13
              Max Indel:             	16000
              Minimum Score Ratio:  	0.56
              Mapping Mode:         	normal
              Reads Used:           	200000	(30000000 bases)
              
              Mapping:          	677.499 seconds.
              Reads/sec:       	295.20
              kBases/sec:      	44.28
              
              
              Pairing data:   	pct reads	num reads 	pct bases	   num bases
              
              mated pairs:     	 88.2500% 	    88250 	 88.2500% 	    26475000
              bad pairs:       	  3.0090% 	     3009 	  3.0090% 	      902700
              insert size avg: 	  347.45
              
              
              Read 1 data:      	pct reads	num reads 	pct bases	   num bases
              
              mapped:          	 92.1860% 	    92186 	 92.1860% 	    13827900
              unambiguous:     	 83.6920% 	    83692 	 83.6920% 	    12553800
              ambiguous:       	  8.4940% 	     8494 	  8.4940% 	     1274100
              low-Q discards:  	  0.0000% 	        0 	  0.0000% 	           0
              
              perfect best site:	 31.9760% 	    31976 	 31.9760% 	     4796400
              semiperfect site:	 32.0660% 	    32066 	 32.0660% 	     4809900
              rescued:         	  4.9400% 	     4940
              
              Match Rate:      	      NA 	       NA 	 86.0733% 	    13594568
              Error Rate:      	 65.2431% 	    60145 	 13.9159% 	     2197904
              Sub Rate:        	 64.5217% 	    59480 	  1.4295% 	      225783
              Del Rate:        	  2.8898% 	     2664 	 12.4494% 	     1966280
              Ins Rate:        	  1.7161% 	     1582 	  0.0370% 	        5841
              N Rate:          	  1.8137% 	     1672 	  0.0108% 	        1708
              
              
              Read 2 data:      	pct reads	num reads 	pct bases	   num bases
              
              mapped:          	 91.4310% 	    91431 	 91.4310% 	    13714650
              unambiguous:     	 83.1120% 	    83112 	 83.1120% 	    12466800
              ambiguous:       	  8.3190% 	     8319 	  8.3190% 	     1247850
              low-Q discards:  	  0.0000% 	        0 	  0.0000% 	           0
              
              perfect best site:	 19.0190% 	    19019 	 19.0190% 	     2852850
              semiperfect site:	 19.0690% 	    19069 	 19.0690% 	     2860350
              rescued:         	  5.3500% 	     5350
              
              Match Rate:      	      NA 	       NA 	 85.6961% 	    13334138
              Error Rate:      	 79.1985% 	    72412 	 14.3034% 	     2225573
              Sub Rate:        	 78.8595% 	    72102 	  2.4028% 	      373878
              Del Rate:        	  2.7977% 	     2558 	 11.8584% 	     1845151
              Ins Rate:        	  1.7839% 	     1631 	  0.0421% 	        6544
              N Rate:          	  0.0022% 	        2 	  0.0006% 	          90
              
              Total time:     	707.069 seconds.
              I ran FastQC previously, but the data looked good ^^

              I'll run reformat.sh in a minute
              Last edited by Brian Bushnell; 11-16-2016, 03:01 PM.

              Comment


              • #8
                The output of reformat:
                Code:
                thomas@streymoy:/mnt/raid0ssd/align/2987$ /home/thomas/src/bbmap/reformat.sh in=S_2987_D16084169_HWM7HCCXX_L1.bbmap.sam mhist=mhist.txt qhist=qhist.txt qahist=qahist.txt aqhist=aqhist.txt bhist=bhist.txt
                java -ea -Xmx200m -cp /home/thomas/src/bbmap/current/ jgi.ReformatReads in=S_2987_D16084169_HWM7HCCXX_L1.bbmap.sam mhist=mhist.txt qhist=qhist.txt qahist=qahist.txt aqhist=aqhist.txt bhist=bhist.txt
                Executing jgi.ReformatReads [in=S_2987_D16084169_HWM7HCCXX_L1.bbmap.sam, mhist=mhist.txt, qhist=qhist.txt, qahist=qahist.txt, aqhist=aqhist.txt, bhist=bhist.txt]
                
                Set match histogram output to mhist.txt
                Set quality histogram output to qhist.txt
                Set quality accuracy histogram output to qahist.txt
                Set average quality histogram output to aqhist.txt
                Set base content histogram output to bhist.txt
                No output stream specified.  To write to stdout, please specify 'out=stdout.fq' or similar.
                Input is being processed as unpaired
                Input:                  	200000 reads          	30000000 bases
                Output:                 	200000 reads (100.00%) 	30000000 bases (100.00%)
                
                Time:                         	1.682 seconds.
                Reads Processed:        200k 	118.87k reads/sec
                Bases Processed:      30000k 	17.83m bases/sec
                
                thomas@streymoy:/mnt/raid0ssd/align/2987$ cat qahist.txt
                #Deviation	13.974
                #DeviationSub	13.433
                #Quality	Match	Sub	Ins	Del	TrueQuality	TrueQualitySub
                0	0	0	0	0		
                1	0	0	0	0		
                2	0	0	0	0		
                3	0	0	0	0		
                4	0	0	0	0		
                5	0	0	0	0		
                6	0	0	0	0		
                7	0	0	0	0		
                8	237898	83920	638	117	5.81	5.85
                9	0	0	0	0		
                10	0	0	0	0		
                11	0	0	0	0		
                12	3584356	802416	5010	1802	7.35	7.38
                13	0	0	0	0		
                14	0	0	0	0		
                15	0	0	0	0		
                16	0	0	0	0		
                17	0	0	0	0		
                18	0	0	0	0		
                19	0	0	0	0		
                20	0	0	0	0		
                21	0	0	0	0		
                22	2384350	102068	2000	709	13.76	13.87
                23	0	0	0	0		
                24	0	0	0	0		
                25	0	0	0	0		
                26	0	0	0	0		
                27	2409378	38450	1226	580	17.84	18.04
                28	0	0	0	0		
                29	0	0	0	0		
                30	0	0	0	0		
                31	0	0	0	0		
                32	4432616	35892	1690	837	20.66	20.95
                33	0	0	0	0		
                34	0	0	0	0		
                35	0	0	0	0		
                36	0	0	0	0		
                37	8475310	39952	3316	1739	22.77	23.29
                38	0	0	0	0		
                39	0	0	0	0		
                40	0	0	0	0		
                41	32333504	96624	10890	6344	24.55	25.26
                
                thomas@streymoy:/mnt/raid0ssd/align/2987$ cat aqhist.txt
                #Quality	count1	fraction1	count2	fraction2
                0	0	0.00000	0	0.00000
                1	0	0.00000	0	0.00000
                2	0	0.00000	0	0.00000
                3	0	0.00000	0	0.00000
                4	0	0.00000	0	0.00000
                5	0	0.00000	0	0.00000
                6	0	0.00000	0	0.00000
                7	0	0.00000	0	0.00000
                8	0	0.00000	0	0.00000
                9	0	0.00000	0	0.00000
                10	0	0.00000	0	0.00000
                11	0	0.00000	0	0.00000
                12	0	0.00000	6	0.00006
                13	4	0.00004	167	0.00167
                14	26	0.00026	1158	0.01158
                15	216	0.00216	2988	0.02988
                16	784	0.00784	4493	0.04493
                17	1768	0.01768	5677	0.05677
                18	3211	0.03211	6747	0.06747
                19	4512	0.04512	7605	0.07605
                20	5495	0.05495	8263	0.08263
                21	5833	0.05833	8564	0.08564
                22	6105	0.06105	8344	0.08344
                23	6694	0.06694	7843	0.07843
                24	6500	0.06500	6787	0.06787
                25	6403	0.06403	6092	0.06092
                26	6269	0.06269	4919	0.04919
                27	5496	0.05496	3985	0.03985
                28	5483	0.05483	3907	0.03907
                29	2294	0.02294	1842	0.01842
                30	6577	0.06577	2520	0.02520
                31	740	0.00740	567	0.00567
                32	5566	0.05566	2658	0.02658
                33	4171	0.04171	701	0.00701
                34	53	0.00053	66	0.00066
                35	212	0.00212	210	0.00210
                36	652	0.00652	440	0.00440
                37	1359	0.01359	733	0.00733
                38	2565	0.02565	907	0.00907
                39	3117	0.03117	793	0.00793
                40	7895	0.07895	1018	0.01018
                
                thomas@streymoy:/mnt/raid0ssd/align/2987$ cat bhist.txt
                #Pos	A	C	G	T	N
                0	0.32827	0.20404	0.26438	0.18520	0.01811
                1	0.30392	0.19942	0.22768	0.26898	0.00000
                2	0.28039	0.22313	0.22764	0.26884	0.00000
                3	0.27765	0.22789	0.23112	0.26334	0.00000
                4	0.27971	0.22454	0.22890	0.26685	0.00000
                5	0.27827	0.22819	0.23082	0.26272	0.00000
                6	0.28159	0.22052	0.23231	0.26558	0.00000
                7	0.27348	0.22661	0.22739	0.27252	0.00000
                8	0.26763	0.22995	0.22617	0.27625	0.00000
                9	0.27684	0.22668	0.22893	0.26755	0.00000
                10	0.27649	0.22634	0.22948	0.26769	0.00000
                11	0.27356	0.22694	0.22579	0.27371	0.00000
                12	0.27507	0.22522	0.22514	0.27457	0.00000
                13	0.27220	0.22470	0.22525	0.27785	0.00000
                14	0.27522	0.22424	0.22676	0.27378	0.00000
                15	0.27275	0.22659	0.22708	0.27358	0.00000
                16	0.27244	0.22592	0.22665	0.27499	0.00000
                17	0.27113	0.23027	0.22385	0.27475	0.00000
                18	0.26951	0.22902	0.22756	0.27391	0.00000
                19	0.27001	0.23068	0.22667	0.27264	0.00000
                20	0.26993	0.23197	0.22703	0.27107	0.00000
                21	0.27192	0.22922	0.22637	0.27249	0.00000
                22	0.27313	0.22580	0.22902	0.27205	0.00000
                23	0.27198	0.22900	0.22863	0.27039	0.00000
                24	0.27238	0.22751	0.22877	0.27134	0.00000
                25	0.26897	0.22731	0.23260	0.27112	0.00000
                26	0.27165	0.23011	0.22802	0.27022	0.00000
                27	0.27224	0.22831	0.23007	0.26938	0.00000
                28	0.26816	0.23055	0.22948	0.27181	0.00000
                29	0.26899	0.23025	0.22803	0.27273	0.00000
                30	0.27264	0.22949	0.23024	0.26763	0.00000
                31	0.27194	0.22638	0.22942	0.27226	0.00000
                32	0.27280	0.22772	0.22815	0.27133	0.00000
                33	0.27131	0.23052	0.22944	0.26873	0.00000
                34	0.26872	0.22976	0.22887	0.27265	0.00000
                35	0.27222	0.22807	0.22950	0.27021	0.00000
                36	0.27298	0.22964	0.22911	0.26827	0.00000
                37	0.27215	0.22356	0.23233	0.27196	0.00000
                38	0.26832	0.22665	0.23519	0.26984	0.00000
                39	0.27029	0.22854	0.23205	0.26912	0.00000
                40	0.27073	0.22900	0.23092	0.26935	0.00000
                41	0.27051	0.22797	0.22942	0.27210	0.00000
                42	0.27099	0.22943	0.22688	0.27270	0.00000
                43	0.26952	0.22993	0.22774	0.27281	0.00000
                44	0.26796	0.22994	0.23029	0.27181	0.00000
                45	0.27322	0.22890	0.22975	0.26813	0.00000
                46	0.27356	0.22386	0.23103	0.27155	0.00000
                47	0.26826	0.23001	0.23071	0.27102	0.00000
                48	0.27147	0.23063	0.22951	0.26839	0.00000
                49	0.27255	0.22977	0.23090	0.26678	0.00000
                50	0.27117	0.22930	0.23240	0.26713	0.00000
                51	0.26898	0.23092	0.23103	0.26907	0.00000
                52	0.27077	0.22984	0.22602	0.27337	0.00000
                53	0.26684	0.23003	0.22932	0.27381	0.00000
                54	0.27021	0.23112	0.22954	0.26913	0.00000
                55	0.27387	0.22939	0.22924	0.26750	0.00000
                56	0.27177	0.22772	0.23029	0.27022	0.00000
                57	0.27217	0.22692	0.23209	0.26882	0.00000
                58	0.27045	0.22817	0.23198	0.26940	0.00000
                59	0.26950	0.23044	0.22862	0.27144	0.00000
                60	0.27133	0.23127	0.23126	0.26614	0.00000
                61	0.27006	0.23050	0.22909	0.27035	0.00000
                62	0.26934	0.22931	0.22994	0.27141	0.00000
                63	0.26948	0.23081	0.23239	0.26732	0.00000
                64	0.27237	0.22629	0.23415	0.26719	0.00000
                65	0.27928	0.21617	0.23990	0.26465	0.00000
                66	0.27022	0.23081	0.23021	0.26876	0.00000
                67	0.27520	0.21916	0.23664	0.26900	0.00000
                68	0.26876	0.22949	0.23154	0.27021	0.00000
                69	0.27141	0.22767	0.23085	0.27007	0.00000
                70	0.26903	0.22933	0.23235	0.26929	0.00000
                71	0.27038	0.23046	0.22925	0.26991	0.00000
                72	0.27295	0.23003	0.22903	0.26799	0.00000
                73	0.27199	0.23207	0.22909	0.26685	0.00000
                74	0.26932	0.23148	0.22954	0.26966	0.00000
                75	0.27237	0.23215	0.22900	0.26648	0.00000
                76	0.27333	0.22793	0.23070	0.26804	0.00000
                77	0.26940	0.23000	0.23058	0.27002	0.00000
                78	0.27009	0.22877	0.23299	0.26815	0.00000
                79	0.27007	0.23113	0.23120	0.26760	0.00000
                80	0.26762	0.22904	0.23434	0.26900	0.00000
                81	0.27045	0.23170	0.22983	0.26802	0.00000
                82	0.27312	0.22880	0.22920	0.26888	0.00000
                83	0.27100	0.23144	0.23063	0.26693	0.00000
                84	0.26756	0.23039	0.23011	0.27194	0.00000
                85	0.27037	0.22925	0.23058	0.26980	0.00000
                86	0.27015	0.22989	0.22976	0.27020	0.00000
                87	0.27330	0.22669	0.23195	0.26806	0.00000
                88	0.27083	0.22873	0.22846	0.27198	0.00000
                89	0.26798	0.23015	0.23275	0.26912	0.00000
                90	0.27004	0.23159	0.23138	0.26699	0.00000
                91	0.27267	0.22777	0.22896	0.27060	0.00000
                92	0.27171	0.22999	0.22849	0.26981	0.00000
                93	0.27015	0.23169	0.23003	0.26813	0.00000
                94	0.26874	0.22739	0.23316	0.27071	0.00000
                95	0.27263	0.22514	0.23281	0.26942	0.00000
                96	0.27388	0.22631	0.23452	0.26529	0.00000
                97	0.28925	0.20214	0.23845	0.27016	0.00000
                98	0.29116	0.19984	0.23380	0.27520	0.00000
                99	0.26931	0.23232	0.22943	0.26894	0.00000
                100	0.27060	0.22836	0.23321	0.26783	0.00000
                101	0.27039	0.23328	0.22934	0.26699	0.00000
                102	0.27181	0.23098	0.22913	0.26808	0.00000
                103	0.27896	0.21731	0.23111	0.27262	0.00000
                104	0.26913	0.23229	0.22940	0.26918	0.00000
                105	0.27275	0.22669	0.23286	0.26770	0.00000
                106	0.27209	0.22950	0.23133	0.26708	0.00000
                107	0.27434	0.22980	0.22740	0.26846	0.00000
                108	0.27087	0.22974	0.23180	0.26759	0.00000
                109	0.27227	0.22637	0.23010	0.27126	0.00000
                110	0.27011	0.22783	0.23299	0.26907	0.00000
                111	0.27242	0.22964	0.22933	0.26861	0.00000
                112	0.27212	0.22943	0.23043	0.26802	0.00000
                113	0.27224	0.23229	0.22869	0.26678	0.00000
                114	0.27134	0.23016	0.23069	0.26781	0.00000
                115	0.27150	0.22977	0.23156	0.26717	0.00000
                116	0.27218	0.22918	0.23024	0.26840	0.00000
                117	0.27402	0.22867	0.22875	0.26856	0.00000
                118	0.27183	0.22849	0.23006	0.26962	0.00000
                119	0.27115	0.22918	0.22894	0.27073	0.00000
                120	0.27428	0.22097	0.23529	0.26946	0.00000
                121	0.27316	0.22928	0.23127	0.26629	0.00000
                122	0.27050	0.22909	0.22953	0.27088	0.00000
                123	0.27703	0.21987	0.23552	0.26758	0.00000
                124	0.27259	0.22432	0.23118	0.27191	0.00000
                125	0.27845	0.21961	0.23356	0.26838	0.00000
                126	0.28921	0.20439	0.23300	0.27340	0.00000
                127	0.30323	0.18830	0.23310	0.27537	0.00000
                128	0.27127	0.22591	0.23349	0.26933	0.00000
                129	0.27068	0.22766	0.23195	0.26971	0.00000
                130	0.28459	0.21733	0.23099	0.26709	0.00000
                131	0.28811	0.20901	0.22902	0.27386	0.00000
                132	0.27265	0.22951	0.23038	0.26746	0.00000
                133	0.28178	0.21763	0.23199	0.26860	0.00000
                134	0.26956	0.23149	0.23057	0.26838	0.00000
                135	0.27333	0.22145	0.23709	0.26813	0.00000
                136	0.27438	0.22738	0.23055	0.26769	0.00000
                137	0.27111	0.22632	0.23038	0.27219	0.00000
                138	0.26867	0.23350	0.22764	0.27019	0.00000
                139	0.27290	0.22537	0.23061	0.27112	0.00000
                140	0.27723	0.22021	0.23218	0.27038	0.00000
                141	0.27412	0.22617	0.22985	0.26986	0.00000
                142	0.27167	0.22740	0.23016	0.27077	0.00000
                143	0.27243	0.22930	0.22904	0.26923	0.00000
                144	0.27837	0.21255	0.23525	0.27383	0.00000
                145	0.27474	0.22596	0.23106	0.26824	0.00000
                146	0.27010	0.23035	0.22900	0.27055	0.00000
                147	0.27549	0.22472	0.23043	0.26936	0.00000
                148	0.27410	0.22212	0.23318	0.27060	0.00000
                149	0.27247	0.22546	0.23262	0.26945	0.00000
                150	0.32612	0.21458	0.27177	0.18753	0.00000
                151	0.31042	0.19927	0.22498	0.26533	0.00000
                152	0.28545	0.22726	0.23028	0.25701	0.00000
                153	0.27659	0.22840	0.23668	0.25833	0.00000
                154	0.28206	0.22517	0.22721	0.26556	0.00000
                155	0.27773	0.22706	0.23384	0.26137	0.00000
                156	0.27628	0.22983	0.22748	0.26641	0.00000
                157	0.27135	0.22837	0.22552	0.27476	0.00000
                158	0.26980	0.22753	0.22718	0.27549	0.00000
                159	0.27342	0.23012	0.22764	0.26882	0.00000
                160	0.27491	0.22706	0.22829	0.26974	0.00000
                161	0.27447	0.22430	0.22454	0.27669	0.00000
                162	0.27263	0.22297	0.22551	0.27889	0.00000
                163	0.27350	0.22527	0.22387	0.27736	0.00000
                164	0.27523	0.22642	0.22364	0.27471	0.00000
                165	0.27048	0.22726	0.22665	0.27561	0.00000
                166	0.27230	0.22753	0.22146	0.27871	0.00000
                167	0.27101	0.22830	0.22433	0.27636	0.00000
                168	0.26858	0.22774	0.22542	0.27826	0.00000
                169	0.27041	0.22783	0.22669	0.27507	0.00000
                170	0.27096	0.22727	0.22750	0.27427	0.00000
                171	0.26964	0.22678	0.22647	0.27711	0.00000
                172	0.27268	0.22753	0.22554	0.27425	0.00000
                173	0.27057	0.22813	0.22752	0.27378	0.00000
                174	0.27008	0.22767	0.22928	0.27297	0.00000
                175	0.27040	0.22823	0.23036	0.27101	0.00000
                176	0.27047	0.22943	0.22762	0.27248	0.00000
                177	0.27035	0.22660	0.22937	0.27368	0.00000
                178	0.26930	0.22711	0.22922	0.27437	0.00000
                179	0.26814	0.22753	0.22976	0.27457	0.00000
                180	0.27071	0.23019	0.22928	0.26982	0.00000
                181	0.27088	0.22685	0.22967	0.27260	0.00000
                182	0.27246	0.22808	0.22909	0.27037	0.00000
                183	0.27188	0.22774	0.22830	0.27208	0.00000
                184	0.26815	0.22932	0.23070	0.27183	0.00000
                185	0.27164	0.22969	0.22803	0.27064	0.00000
                186	0.26973	0.23017	0.22986	0.27024	0.00000
                187	0.27615	0.22402	0.23021	0.26962	0.00000
                188	0.27290	0.22529	0.23091	0.27090	0.00000
                189	0.26534	0.22834	0.23266	0.27366	0.00000
                190	0.27117	0.22749	0.23173	0.26961	0.00000
                191	0.26858	0.23136	0.22989	0.27017	0.00000
                192	0.26664	0.23032	0.23096	0.27208	0.00000
                193	0.26781	0.23187	0.22730	0.27302	0.00000
                194	0.26772	0.23034	0.22820	0.27374	0.00000
                195	0.27347	0.22649	0.23260	0.26744	0.00000
                196	0.26876	0.22929	0.23040	0.27155	0.00000
                197	0.27079	0.22992	0.22676	0.27253	0.00000
                198	0.26758	0.22721	0.23272	0.27249	0.00000
                199	0.26657	0.22857	0.23456	0.27030	0.00000
                200	0.26576	0.23405	0.23013	0.27006	0.00000
                201	0.27228	0.23145	0.22898	0.26729	0.00000
                202	0.26943	0.22910	0.22724	0.27423	0.00000
                203	0.27018	0.22985	0.22786	0.27211	0.00000
                204	0.26584	0.23127	0.23206	0.27083	0.00000
                205	0.27250	0.22831	0.22957	0.26962	0.00000
                206	0.27088	0.23295	0.22733	0.26884	0.00000
                207	0.26785	0.23238	0.22868	0.27109	0.00000
                208	0.26983	0.23220	0.22985	0.26812	0.00000
                209	0.26860	0.23025	0.22970	0.27145	0.00000
                210	0.27500	0.22957	0.22879	0.26664	0.00000
                211	0.27048	0.23139	0.22736	0.27077	0.00000
                212	0.27053	0.22958	0.22893	0.27096	0.00000
                213	0.26953	0.22719	0.23109	0.27219	0.00000
                214	0.26912	0.23009	0.22958	0.27121	0.00000
                215	0.27074	0.22986	0.22899	0.27041	0.00000
                216	0.27049	0.22996	0.23117	0.26838	0.00000
                217	0.27139	0.22659	0.23076	0.27126	0.00000
                218	0.26886	0.22965	0.22963	0.27186	0.00000
                219	0.26735	0.23075	0.23147	0.27043	0.00000
                220	0.27742	0.22328	0.23113	0.26817	0.00000
                221	0.26616	0.23229	0.23125	0.27030	0.00000
                222	0.27113	0.23017	0.22978	0.26892	0.00000
                223	0.26904	0.22846	0.22996	0.27254	0.00000
                224	0.26813	0.22838	0.23230	0.27119	0.00000
                225	0.26833	0.22884	0.23211	0.27072	0.00000
                226	0.27023	0.22838	0.23095	0.27044	0.00000
                227	0.26752	0.23099	0.23052	0.27097	0.00000
                228	0.27312	0.22571	0.23028	0.27089	0.00000
                229	0.26923	0.23052	0.23182	0.26843	0.00000
                230	0.27493	0.22414	0.23120	0.26973	0.00000
                231	0.27031	0.22843	0.23280	0.26846	0.00000
                232	0.26833	0.22895	0.23147	0.27125	0.00000
                233	0.26865	0.23203	0.23038	0.26894	0.00000
                234	0.27184	0.23083	0.22666	0.27067	0.00000
                235	0.27066	0.22850	0.22842	0.27242	0.00000
                236	0.27081	0.23015	0.22863	0.27041	0.00000
                237	0.27116	0.22871	0.22889	0.27124	0.00000
                238	0.27132	0.22697	0.23337	0.26834	0.00000
                239	0.27027	0.23221	0.22854	0.26898	0.00000
                240	0.26977	0.22856	0.23095	0.27072	0.00000
                241	0.27644	0.22082	0.23335	0.26939	0.00000
                242	0.26936	0.23217	0.22927	0.26920	0.00000
                243	0.27005	0.23116	0.22836	0.27043	0.00000
                244	0.26459	0.23178	0.23197	0.27166	0.00000
                245	0.27632	0.22462	0.22872	0.27034	0.00000
                246	0.27227	0.22924	0.22836	0.27013	0.00000
                247	0.27427	0.22343	0.22948	0.27282	0.00000
                248	0.27354	0.22390	0.23237	0.27019	0.00000
                249	0.26674	0.23549	0.23077	0.26700	0.00000
                250	0.27080	0.23120	0.23052	0.26748	0.00000
                251	0.26952	0.22716	0.23247	0.27085	0.00000
                252	0.26907	0.22859	0.23239	0.26995	0.00000
                253	0.26882	0.22748	0.23114	0.27256	0.00000
                254	0.26900	0.22678	0.23037	0.27385	0.00000
                255	0.28935	0.20645	0.23521	0.26899	0.00000
                256	0.26891	0.23217	0.23252	0.26640	0.00000
                257	0.27367	0.22655	0.22904	0.27074	0.00000
                258	0.26963	0.23065	0.23080	0.26892	0.00000
                259	0.26843	0.22966	0.23061	0.27130	0.00000
                260	0.28209	0.21145	0.23647	0.26999	0.00000
                261	0.27401	0.22252	0.23309	0.27038	0.00000
                262	0.27165	0.22579	0.23021	0.27235	0.00000
                263	0.27193	0.22318	0.23358	0.27131	0.00000
                264	0.26915	0.22913	0.23094	0.27078	0.00000
                265	0.27499	0.22055	0.23147	0.27299	0.00000
                266	0.27702	0.22096	0.23422	0.26780	0.00000
                267	0.27611	0.22490	0.23118	0.26781	0.00000
                268	0.29368	0.19576	0.23804	0.27252	0.00000
                269	0.26978	0.23065	0.23012	0.26945	0.00000
                270	0.27165	0.23254	0.22858	0.26723	0.00000
                271	0.29649	0.20093	0.23510	0.26748	0.00000
                272	0.27161	0.23003	0.22748	0.27088	0.00000
                273	0.28829	0.19364	0.24375	0.27432	0.00000
                274	0.27385	0.22282	0.23062	0.27271	0.00000
                275	0.29731	0.19999	0.23726	0.26544	0.00000
                276	0.26898	0.23117	0.23053	0.26932	0.00000
                277	0.27403	0.22878	0.22855	0.26864	0.00000
                278	0.27463	0.21839	0.23421	0.27277	0.00000
                279	0.27117	0.22793	0.23116	0.26974	0.00000
                280	0.28182	0.21772	0.23228	0.26818	0.00000
                281	0.27505	0.22532	0.23086	0.26877	0.00000
                282	0.27536	0.22582	0.23094	0.26788	0.00000
                283	0.26968	0.23058	0.22979	0.26995	0.00000
                284	0.27435	0.22835	0.22841	0.26889	0.00000
                285	0.27612	0.22249	0.23218	0.26921	0.00000
                286	0.27283	0.23093	0.22874	0.26750	0.00000
                287	0.27377	0.22836	0.23073	0.26714	0.00000
                288	0.29024	0.20134	0.23899	0.26943	0.00000
                289	0.27019	0.22671	0.22898	0.27412	0.00000
                290	0.27628	0.21531	0.23535	0.27306	0.00000
                291	0.29074	0.20599	0.23145	0.27182	0.00000
                292	0.29742	0.19889	0.23883	0.26486	0.00000
                293	0.26962	0.22837	0.22945	0.27256	0.00000
                294	0.27096	0.22329	0.23388	0.27187	0.00000
                295	0.27212	0.22616	0.23318	0.26854	0.00000
                296	0.27330	0.22368	0.22992	0.27310	0.00000
                297	0.29481	0.20695	0.23231	0.26593	0.00000
                298	0.27959	0.21269	0.23453	0.27319	0.00000
                299	0.26980	0.22626	0.23194	0.27200	0.00000
                
                thomas@streymoy:/mnt/raid0ssd/align/2987$ cat mhist.txt
                #BaseNum	Match1	Sub1	Del1	Ins1	N1	Other1	Match2	Sub2	Del2	Ins2	N2	Other2
                1	0.97566	0.00620	0.00000	0.00000	0.01812	0.00002	0.99186	0.00814	0.00000	0.00000	0.00000	0.00000
                2	0.99508	0.00490	0.00000	0.00000	0.00000	0.00002	0.99237	0.00763	0.00000	0.00000	0.00000	0.00000
                3	0.99542	0.00455	0.00000	0.00001	0.00000	0.00002	0.99405	0.00593	0.00000	0.00002	0.00000	0.00000
                4	0.99549	0.00445	0.00002	0.00004	0.00000	0.00002	0.99349	0.00643	0.00004	0.00008	0.00000	0.00000
                5	0.99583	0.00404	0.00004	0.00011	0.00000	0.00002	0.99426	0.00560	0.00003	0.00014	0.00000	0.00000
                6	0.99582	0.00398	0.00010	0.00017	0.00000	0.00002	0.99533	0.00454	0.00011	0.00013	0.00000	0.00000
                7	0.98460	0.01519	0.00012	0.00020	0.00000	0.00002	0.99400	0.00579	0.00007	0.00022	0.00000	0.00000
                8	0.99608	0.00371	0.00010	0.00018	0.00000	0.00002	0.99443	0.00534	0.00010	0.00023	0.00000	0.00000
                9	0.99586	0.00394	0.00011	0.00018	0.00000	0.00002	0.99451	0.00524	0.00023	0.00025	0.00000	0.00000
                10	0.99599	0.00376	0.00015	0.00023	0.00000	0.00002	0.99330	0.00639	0.00017	0.00032	0.00000	0.00000
                11	0.99614	0.00358	0.00017	0.00026	0.00000	0.00002	0.99264	0.00705	0.00021	0.00031	0.00000	0.00000
                12	0.99629	0.00347	0.00016	0.00022	0.00000	0.00002	0.99421	0.00550	0.00016	0.00028	0.00000	0.00000
                13	0.99561	0.00419	0.00014	0.00018	0.00000	0.00002	0.99167	0.00806	0.00017	0.00027	0.00000	0.00000
                14	0.99586	0.00387	0.00015	0.00025	0.00000	0.00002	0.99393	0.00581	0.00012	0.00026	0.00000	0.00000
                15	0.99583	0.00387	0.00022	0.00027	0.00000	0.00002	0.98883	0.01083	0.00016	0.00034	0.00000	0.00000
                16	0.99531	0.00444	0.00012	0.00023	0.00000	0.00002	0.99483	0.00487	0.00012	0.00031	0.00000	0.00000
                17	0.99604	0.00366	0.00023	0.00028	0.00000	0.00002	0.98810	0.01158	0.00017	0.00032	0.00000	0.00000
                18	0.99640	0.00330	0.00014	0.00028	0.00000	0.00002	0.99410	0.00550	0.00022	0.00039	0.00000	0.00000
                19	0.99671	0.00305	0.00014	0.00022	0.00000	0.00002	0.97584	0.02381	0.00023	0.00035	0.00000	0.00000
                20	0.99616	0.00354	0.00018	0.00030	0.00000	0.00000	0.99394	0.00574	0.00017	0.00032	0.00000	0.00000
                21	0.99626	0.00342	0.00016	0.00033	0.00000	0.00000	0.99207	0.00756	0.00020	0.00037	0.00000	0.00000
                22	0.99644	0.00322	0.00022	0.00034	0.00000	0.00000	0.99495	0.00470	0.00015	0.00035	0.00000	0.00000
                23	0.99642	0.00333	0.00016	0.00025	0.00000	0.00000	0.99386	0.00571	0.00019	0.00043	0.00000	0.00000
                24	0.99681	0.00299	0.00022	0.00020	0.00000	0.00000	0.99213	0.00745	0.00016	0.00043	0.00000	0.00000
                25	0.99627	0.00353	0.00027	0.00021	0.00000	0.00000	0.99206	0.00750	0.00024	0.00044	0.00000	0.00000
                26	0.99644	0.00334	0.00020	0.00022	0.00000	0.00000	0.99110	0.00850	0.00016	0.00040	0.00000	0.00000
                27	0.99625	0.00359	0.00021	0.00016	0.00000	0.00000	0.99447	0.00505	0.00015	0.00048	0.00000	0.00000
                28	0.99523	0.00460	0.00020	0.00017	0.00000	0.00000	0.99332	0.00622	0.00015	0.00046	0.00000	0.00000
                29	0.99628	0.00348	0.00029	0.00024	0.00000	0.00000	0.99417	0.00535	0.00025	0.00048	0.00000	0.00000
                30	0.99606	0.00374	0.00025	0.00020	0.00000	0.00000	0.98913	0.01047	0.00015	0.00040	0.00000	0.00000
                31	0.99543	0.00438	0.00020	0.00018	0.00000	0.00000	0.99340	0.00617	0.00015	0.00043	0.00000	0.00000
                32	0.99623	0.00354	0.00023	0.00024	0.00000	0.00000	0.98574	0.01384	0.00019	0.00043	0.00000	0.00000
                33	0.99570	0.00407	0.00022	0.00023	0.00000	0.00000	0.99090	0.00872	0.00026	0.00038	0.00000	0.00000
                34	0.99611	0.00362	0.00028	0.00027	0.00000	0.00000	0.99332	0.00634	0.00028	0.00034	0.00000	0.00000
                35	0.99573	0.00397	0.00017	0.00030	0.00000	0.00000	0.99280	0.00680	0.00025	0.00039	0.00000	0.00000
                36	0.99588	0.00389	0.00033	0.00023	0.00000	0.00000	0.99363	0.00595	0.00016	0.00042	0.00000	0.00000
                37	0.99586	0.00381	0.00017	0.00034	0.00000	0.00000	0.99400	0.00557	0.00019	0.00044	0.00000	0.00000
                38	0.99473	0.00487	0.00022	0.00040	0.00000	0.00000	0.98006	0.01952	0.00016	0.00042	0.00000	0.00000
                39	0.99356	0.00601	0.00029	0.00043	0.00000	0.00000	0.98779	0.01184	0.00028	0.00036	0.00000	0.00000
                40	0.99383	0.00576	0.00021	0.00041	0.00000	0.00000	0.98729	0.01226	0.00017	0.00045	0.00000	0.00000
                41	0.99579	0.00382	0.00041	0.00039	0.00000	0.00000	0.98834	0.01121	0.00013	0.00045	0.00000	0.00000
                42	0.99611	0.00354	0.00014	0.00036	0.00000	0.00000	0.99343	0.00615	0.00016	0.00043	0.00000	0.00000
                43	0.99544	0.00414	0.00016	0.00041	0.00000	0.00000	0.99267	0.00689	0.00027	0.00044	0.00000	0.00000
                44	0.99573	0.00387	0.00029	0.00040	0.00000	0.00000	0.99477	0.00486	0.00033	0.00037	0.00000	0.00000
                45	0.99559	0.00404	0.00033	0.00038	0.00000	0.00000	0.99429	0.00529	0.00023	0.00042	0.00000	0.00000
                46	0.99592	0.00371	0.00024	0.00037	0.00000	0.00000	0.98494	0.01457	0.00032	0.00049	0.00000	0.00000
                47	0.99538	0.00415	0.00023	0.00047	0.00000	0.00000	0.99407	0.00550	0.00019	0.00043	0.00000	0.00000
                48	0.99550	0.00411	0.00030	0.00039	0.00000	0.00000	0.99537	0.00425	0.00035	0.00037	0.00000	0.00000
                49	0.99542	0.00426	0.00037	0.00031	0.00000	0.00000	0.98926	0.01039	0.00027	0.00035	0.00000	0.00000
                50	0.99590	0.00370	0.00022	0.00040	0.00000	0.00000	0.99167	0.00803	0.00026	0.00031	0.00000	0.00000
                51	0.99553	0.00402	0.00022	0.00044	0.00000	0.00000	0.99480	0.00490	0.00021	0.00030	0.00000	0.00000
                52	0.99600	0.00361	0.00026	0.00039	0.00000	0.00000	0.99310	0.00662	0.00038	0.00028	0.00000	0.00000
                53	0.99550	0.00412	0.00027	0.00038	0.00000	0.00000	0.99150	0.00816	0.00021	0.00034	0.00000	0.00000
                54	0.99552	0.00402	0.00026	0.00046	0.00000	0.00000	0.98929	0.01039	0.00032	0.00032	0.00000	0.00000
                55	0.99486	0.00481	0.00035	0.00034	0.00000	0.00000	0.99318	0.00652	0.00020	0.00031	0.00000	0.00000
                56	0.99481	0.00473	0.00040	0.00046	0.00000	0.00000	0.98662	0.01304	0.00034	0.00034	0.00000	0.00000
                57	0.99542	0.00414	0.00017	0.00043	0.00000	0.00000	0.99094	0.00877	0.00026	0.00028	0.00000	0.00000
                58	0.99066	0.00895	0.00025	0.00039	0.00000	0.00000	0.99371	0.00607	0.00027	0.00022	0.00000	0.00000
                59	0.99553	0.00407	0.00031	0.00040	0.00000	0.00000	0.98901	0.01077	0.00023	0.00022	0.00000	0.00000
                60	0.99484	0.00477	0.00041	0.00039	0.00000	0.00000	0.98795	0.01177	0.00035	0.00028	0.00000	0.00000
                61	0.99537	0.00418	0.00025	0.00046	0.00000	0.00000	0.98397	0.01575	0.00030	0.00028	0.00000	0.00000
                62	0.99559	0.00401	0.00024	0.00040	0.00000	0.00000	0.98160	0.01805	0.00022	0.00035	0.00000	0.00000
                63	0.99565	0.00393	0.00021	0.00042	0.00000	0.00000	0.99136	0.00830	0.00025	0.00034	0.00000	0.00000
                64	0.99437	0.00521	0.00034	0.00042	0.00000	0.00000	0.99375	0.00595	0.00022	0.00030	0.00000	0.00000
                65	0.97998	0.01958	0.00018	0.00044	0.00000	0.00000	0.99180	0.00792	0.00023	0.00028	0.00000	0.00000
                66	0.91732	0.08206	0.00042	0.00062	0.00000	0.00000	0.99308	0.00664	0.00026	0.00028	0.00000	0.00000
                67	0.99549	0.00398	0.00035	0.00053	0.00000	0.00000	0.99305	0.00664	0.00025	0.00031	0.00000	0.00000
                68	0.95608	0.04334	0.00033	0.00059	0.00000	0.00000	0.98340	0.01633	0.00022	0.00027	0.00000	0.00000
                69	0.98499	0.01454	0.00024	0.00048	0.00000	0.00000	0.99218	0.00750	0.00026	0.00032	0.00000	0.00000
                70	0.99522	0.00428	0.00029	0.00050	0.00000	0.00000	0.98723	0.01247	0.00030	0.00031	0.00000	0.00000
                71	0.99473	0.00482	0.00022	0.00046	0.00000	0.00000	0.97425	0.02543	0.00021	0.00032	0.00000	0.00000
                72	0.99142	0.00820	0.00029	0.00038	0.00000	0.00000	0.99109	0.00856	0.00027	0.00035	0.00000	0.00000
                73	0.99506	0.00445	0.00029	0.00049	0.00000	0.00000	0.99165	0.00804	0.00039	0.00031	0.00000	0.00000
                74	0.99449	0.00512	0.00035	0.00039	0.00000	0.00000	0.99257	0.00708	0.00030	0.00035	0.00000	0.00000
                75	0.99523	0.00432	0.00023	0.00046	0.00000	0.00000	0.99181	0.00785	0.00028	0.00034	0.00000	0.00000
                76	0.99578	0.00381	0.00025	0.00041	0.00000	0.00000	0.99295	0.00668	0.00027	0.00037	0.00000	0.00000
                77	0.99539	0.00421	0.00030	0.00040	0.00000	0.00000	0.99268	0.00695	0.00022	0.00037	0.00000	0.00000
                78	0.99542	0.00420	0.00026	0.00038	0.00000	0.00000	0.99285	0.00678	0.00023	0.00037	0.00000	0.00000
                79	0.99490	0.00472	0.00030	0.00038	0.00000	0.00000	0.98731	0.01234	0.00027	0.00035	0.00000	0.00000
                80	0.99510	0.00446	0.00017	0.00044	0.00000	0.00000	0.99313	0.00647	0.00017	0.00039	0.00000	0.00000
                81	0.99345	0.00620	0.00024	0.00035	0.00000	0.00000	0.97363	0.02588	0.00030	0.00048	0.00000	0.00001
                82	0.99504	0.00468	0.00030	0.00028	0.00000	0.00000	0.98525	0.01423	0.00031	0.00051	0.00000	0.00001
                83	0.99541	0.00436	0.00030	0.00023	0.00000	0.00000	0.99368	0.00590	0.00028	0.00042	0.00000	0.00001
                84	0.99538	0.00437	0.00023	0.00025	0.00000	0.00000	0.99234	0.00722	0.00013	0.00043	0.00000	0.00001
                85	0.99572	0.00399	0.00022	0.00029	0.00000	0.00000	0.99257	0.00698	0.00019	0.00044	0.00000	0.00001
                86	0.99467	0.00504	0.00028	0.00028	0.00000	0.00000	0.99321	0.00635	0.00026	0.00043	0.00000	0.00001
                87	0.99547	0.00422	0.00016	0.00031	0.00000	0.00000	0.99310	0.00647	0.00032	0.00042	0.00000	0.00001
                88	0.99298	0.00658	0.00027	0.00043	0.00000	0.00000	0.99327	0.00631	0.00022	0.00040	0.00000	0.00001
                89	0.99512	0.00448	0.00030	0.00040	0.00000	0.00000	0.98948	0.01012	0.00021	0.00039	0.00000	0.00001
                90	0.99146	0.00815	0.00020	0.00039	0.00000	0.00000	0.99180	0.00783	0.00015	0.00036	0.00000	0.00001
                91	0.99409	0.00553	0.00030	0.00038	0.00000	0.00000	0.98969	0.00997	0.00034	0.00033	0.00000	0.00001
                92	0.98019	0.01946	0.00024	0.00035	0.00000	0.00000	0.95707	0.04255	0.00019	0.00037	0.00000	0.00001
                93	0.98515	0.01445	0.00017	0.00040	0.00000	0.00000	0.99215	0.00749	0.00024	0.00035	0.00000	0.00001
                94	0.99509	0.00452	0.00021	0.00039	0.00000	0.00000	0.98527	0.01422	0.00013	0.00050	0.00000	0.00001
                95	0.99122	0.00828	0.00020	0.00050	0.00000	0.00000	0.99221	0.00727	0.00026	0.00050	0.00000	0.00001
                96	0.97739	0.02202	0.00021	0.00059	0.00000	0.00000	0.96841	0.03108	0.00030	0.00049	0.00000	0.00001
                97	0.98276	0.01660	0.00034	0.00064	0.00000	0.00000	0.98226	0.01727	0.00034	0.00046	0.00000	0.00001
                98	0.84955	0.14963	0.00031	0.00081	0.00000	0.00000	0.98163	0.01781	0.00019	0.00056	0.00000	0.00001
                99	0.82360	0.17572	0.00041	0.00068	0.00000	0.00000	0.96654	0.03300	0.00025	0.00045	0.00000	0.00001
                100	0.99492	0.00459	0.00039	0.00049	0.00000	0.00000	0.99159	0.00793	0.00030	0.00047	0.00000	0.00001
                101	0.99472	0.00482	0.00023	0.00047	0.00000	0.00000	0.98767	0.01189	0.00015	0.00043	0.00000	0.00001
                102	0.99478	0.00473	0.00030	0.00049	0.00000	0.00000	0.98350	0.01603	0.00019	0.00046	0.00000	0.00001
                103	0.99471	0.00481	0.00018	0.00049	0.00000	0.00000	0.99057	0.00890	0.00028	0.00051	0.00000	0.00001
                104	0.93356	0.06578	0.00026	0.00066	0.00000	0.00000	0.99172	0.00773	0.00017	0.00054	0.00000	0.00001
                105	0.99090	0.00853	0.00022	0.00057	0.00000	0.00000	0.98131	0.01806	0.00030	0.00062	0.00000	0.00001
                106	0.98194	0.01762	0.00033	0.00044	0.00000	0.00000	0.87054	0.12874	0.00033	0.00071	0.00000	0.00001
                107	0.99485	0.00468	0.00025	0.00048	0.00000	0.00000	0.99005	0.00932	0.00034	0.00062	0.00000	0.00001
                108	0.99223	0.00725	0.00044	0.00052	0.00000	0.00000	0.97828	0.02113	0.00027	0.00058	0.00000	0.00001
                109	0.99346	0.00602	0.00018	0.00052	0.00000	0.00000	0.99171	0.00771	0.00020	0.00057	0.00000	0.00001
                110	0.97853	0.02090	0.00030	0.00056	0.00000	0.00000	0.98890	0.01045	0.00032	0.00065	0.00000	0.00001
                111	0.97698	0.02248	0.00016	0.00054	0.00000	0.00000	0.91880	0.08057	0.00026	0.00061	0.00000	0.00001
                112	0.98946	0.01000	0.00025	0.00054	0.00000	0.00000	0.97366	0.02576	0.00023	0.00057	0.00000	0.00001
                113	0.98890	0.01052	0.00023	0.00057	0.00000	0.00000	0.97771	0.02168	0.00023	0.00060	0.00000	0.00001
                114	0.99299	0.00647	0.00029	0.00054	0.00000	0.00000	0.96454	0.03484	0.00019	0.00061	0.00000	0.00001
                115	0.99249	0.00691	0.00017	0.00060	0.00000	0.00000	0.98981	0.00964	0.00020	0.00055	0.00000	0.00001
                116	0.98896	0.01047	0.00035	0.00057	0.00000	0.00000	0.95404	0.04531	0.00024	0.00063	0.00000	0.00001
                117	0.99271	0.00676	0.00031	0.00053	0.00000	0.00000	0.95197	0.04736	0.00022	0.00066	0.00000	0.00001
                118	0.99140	0.00806	0.00021	0.00054	0.00000	0.00000	0.97469	0.02455	0.00021	0.00074	0.00000	0.00001
                119	0.99183	0.00756	0.00034	0.00061	0.00000	0.00000	0.80309	0.19594	0.00030	0.00096	0.00000	0.00001
                120	0.99145	0.00794	0.00023	0.00061	0.00000	0.00000	0.98613	0.01310	0.00021	0.00075	0.00000	0.00001
                121	0.94888	0.05043	0.00034	0.00069	0.00000	0.00000	0.98562	0.01358	0.00028	0.00079	0.00000	0.00001
                122	0.98781	0.01154	0.00022	0.00065	0.00000	0.00000	0.84046	0.15860	0.00028	0.00093	0.00000	0.00001
                123	0.99124	0.00808	0.00022	0.00068	0.00000	0.00000	0.98717	0.01188	0.00023	0.00094	0.00000	0.00001
                124	0.94877	0.05046	0.00018	0.00077	0.00000	0.00000	0.79040	0.20846	0.00031	0.00113	0.00000	0.00001
                125	0.97406	0.02519	0.00021	0.00075	0.00000	0.00000	0.96055	0.03843	0.00021	0.00101	0.00000	0.00001
                126	0.94406	0.05512	0.00018	0.00082	0.00000	0.00000	0.74333	0.25541	0.00023	0.00126	0.00000	0.00001
                127	0.85160	0.14748	0.00034	0.00091	0.00000	0.00000	0.98539	0.01366	0.00025	0.00094	0.00000	0.00001
                128	0.76511	0.23378	0.00030	0.00112	0.00000	0.00000	0.98638	0.01275	0.00026	0.00085	0.00000	0.00001
                129	0.97528	0.02386	0.00035	0.00086	0.00000	0.00000	0.93939	0.05977	0.00019	0.00083	0.00000	0.00001
                130	0.97641	0.02275	0.00027	0.00085	0.00000	0.00000	0.98075	0.01843	0.00020	0.00081	0.00000	0.00001
                131	0.94036	0.05878	0.00015	0.00086	0.00000	0.00000	0.93887	0.06036	0.00020	0.00074	0.00000	0.00002
                132	0.89062	0.10863	0.00014	0.00075	0.00000	0.00000	0.95922	0.03998	0.00013	0.00079	0.00000	0.00002
                133	0.97699	0.02234	0.00027	0.00067	0.00000	0.00000	0.95854	0.04060	0.00019	0.00084	0.00000	0.00002
                134	0.94269	0.05654	0.00014	0.00077	0.00000	0.00000	0.98364	0.01552	0.00020	0.00082	0.00000	0.00002
                135	0.99074	0.00859	0.00024	0.00067	0.00000	0.00000	0.97388	0.02526	0.00012	0.00083	0.00000	0.00002
                136	0.96420	0.03518	0.00022	0.00062	0.00000	0.00000	0.95608	0.04299	0.00015	0.00091	0.00000	0.00002
                137	0.99090	0.00852	0.00020	0.00059	0.00000	0.00000	0.98405	0.01513	0.00028	0.00080	0.00000	0.00002
                138	0.98870	0.01071	0.00021	0.00060	0.00000	0.00000	0.98028	0.01879	0.00020	0.00091	0.00000	0.00002
                139	0.98817	0.01118	0.00014	0.00065	0.00000	0.00000	0.78277	0.21598	0.00022	0.00124	0.00000	0.00002
                140	0.97985	0.01955	0.00022	0.00061	0.00000	0.00000	0.96934	0.02966	0.00013	0.00097	0.00000	0.00002
                141	0.95376	0.04569	0.00018	0.00055	0.00000	0.00000	0.91901	0.07989	0.00026	0.00108	0.00000	0.00002
                142	0.98223	0.01720	0.00015	0.00056	0.00000	0.00000	0.77525	0.22355	0.00027	0.00118	0.00000	0.00002
                143	0.98512	0.01434	0.00008	0.00054	0.00000	0.00000	0.70705	0.29186	0.00024	0.00107	0.00000	0.00002
                144	0.98465	0.01487	0.00012	0.00048	0.00000	0.00000	0.97614	0.02311	0.00016	0.00073	0.00000	0.00002
                145	0.91930	0.08031	0.00007	0.00039	0.00000	0.00000	0.97360	0.02581	0.00017	0.00057	0.00000	0.00002
                146	0.97839	0.02128	0.00010	0.00033	0.00000	0.00000	0.97068	0.02887	0.00017	0.00043	0.00000	0.00002
                147	0.99035	0.00947	0.00008	0.00018	0.00000	0.00000	0.96892	0.03071	0.00013	0.00035	0.00000	0.00002
                148	0.98619	0.01366	0.00004	0.00015	0.00000	0.00000	0.73731	0.26231	0.00007	0.00036	0.00000	0.00002
                149	0.97721	0.02279	0.00000	0.00000	0.00000	0.00000	0.89943	0.10055	0.00000	0.00000	0.00000	0.00002
                150	0.96510	0.03490	0.00000	0.00000	0.00000	0.00000	0.97845	0.02152	0.00000	0.00000	0.00000	0.00002
                
                thomas@streymoy:/mnt/raid0ssd/align/2987$ cat qhist.txt
                #BaseNum	Read1_linear	Read1_log	Read1_measured	Read2_linear	Read2_log	Read2_measured
                1	30.619	17.676	21.993	30.805	24.316	20.895
                2	31.622	28.335	23.095	30.791	24.246	21.172
                3	35.050	27.981	23.414	34.429	26.157	22.255
                4	36.147	30.481	23.456	34.889	25.157	21.837
                5	36.382	31.270	23.781	35.538	26.656	22.385
                6	39.430	30.419	23.714	39.046	28.292	23.206
                7	37.072	23.981	18.096	38.902	27.429	22.168
                8	39.546	31.539	23.988	38.766	26.972	22.468
                9	39.615	31.064	23.736	38.816	27.211	22.426
                10	39.610	30.928	23.826	38.621	26.530	21.624
                11	39.948	32.697	23.965	38.332	25.962	21.210
                12	39.610	30.600	24.144	38.797	27.070	22.255
                13	39.290	29.385	23.456	37.965	25.117	20.701
                14	39.711	31.422	23.692	38.656	26.849	22.083
                15	39.609	30.644	23.604	37.640	24.557	19.457
                16	39.596	30.319	23.202	38.995	27.974	22.762
                17	39.932	32.591	23.803	37.296	23.975	19.181
                18	39.862	31.998	24.294	38.357	26.363	22.137
                19	39.876	32.090	24.677	35.679	21.921	16.128
                20	39.712	31.325	23.953	37.805	25.616	22.052
                21	39.774	31.708	24.084	37.568	24.715	20.901
                22	39.791	31.824	24.231	38.467	26.489	22.835
                23	39.708	31.476	24.268	38.532	26.531	21.992
                24	39.692	31.262	24.677	38.252	25.779	20.948
                25	39.538	30.701	23.976	37.974	25.323	20.872
                26	39.982	32.287	24.256	37.459	24.385	20.425
                27	39.679	30.350	24.024	38.540	26.831	22.451
                28	39.168	28.404	23.038	38.295	25.995	21.652
                29	39.413	29.464	23.965	38.499	26.510	22.160
                30	39.550	30.005	23.781	37.227	24.056	19.576
                31	39.732	30.712	23.222	38.193	25.944	21.708
                32	39.826	31.097	23.976	36.299	22.964	18.402
                33	39.647	30.164	23.456	37.139	24.236	20.286
                34	39.596	29.887	23.792	37.784	25.205	21.570
                35	39.578	29.992	23.519	37.867	25.249	21.279
                36	39.536	29.623	23.519	38.218	25.993	21.851
                37	39.382	29.534	23.648	38.383	26.217	22.083
                38	38.978	27.868	22.605	35.978	22.311	16.967
                39	38.499	26.337	21.716	36.393	23.016	19.034
                40	38.509	26.645	21.953	36.400	22.968	18.899
                41	39.186	28.802	23.353	36.618	23.343	19.285
                42	39.354	29.072	23.941	37.467	24.754	21.715
                43	39.208	28.614	23.262	37.609	24.906	21.191
                44	39.400	29.481	23.404	38.340	26.377	22.552
                45	39.480	29.442	23.242	38.393	26.230	22.263
                46	39.643	30.100	23.648	36.489	23.006	18.131
                47	39.605	30.055	23.144	38.033	25.841	22.137
                48	39.438	29.354	23.183	38.667	27.075	23.031
                49	39.015	27.901	23.057	37.191	23.899	19.581
                50	39.298	28.998	23.648	37.571	24.711	20.657
                51	39.528	29.678	23.292	38.519	26.747	22.674
                52	39.577	29.808	23.703	38.216	25.778	21.376
                53	39.201	28.474	23.212	37.548	24.652	20.602
                54	39.280	28.743	23.242	36.986	23.651	19.576
                55	39.383	29.062	22.605	37.724	25.181	21.536
                56	39.361	28.980	22.529	36.567	23.156	18.628
                57	39.130	28.200	23.232	37.382	24.477	20.307
                58	37.702	24.839	20.182	37.790	25.283	21.829
                59	38.962	28.276	23.202	37.230	24.115	19.499
                60	38.912	27.676	22.537	36.488	23.349	19.065
                61	39.248	28.821	23.115	35.843	22.433	17.870
                62	39.367	29.109	23.322	35.843	22.440	17.301
                63	39.436	29.331	23.414	36.855	24.203	20.510
                64	38.925	27.659	22.243	37.381	25.000	21.895
                65	36.357	22.881	16.945	37.388	24.617	20.740
                66	31.060	18.428	10.804	37.747	25.169	21.435
                67	37.777	27.353	23.134	37.773	25.100	21.429
                68	33.096	19.846	13.541	36.095	22.575	17.741
                69	34.906	21.864	18.167	37.055	24.241	20.924
                70	38.019	27.345	22.944	36.371	23.008	18.837
                71	38.479	27.175	22.605	34.383	20.988	15.858
                72	37.332	24.550	20.519	36.058	23.292	20.368
                73	38.666	27.537	22.816	37.150	24.528	20.585
                74	38.794	27.304	22.323	37.423	24.670	21.123
                75	39.121	28.484	23.010	37.277	24.331	20.718
                76	38.616	28.443	23.498	35.816	24.376	21.350
                77	38.661	27.896	23.086	36.922	24.531	21.229
                78	38.795	27.811	23.153	37.423	24.878	21.318
                79	38.994	28.094	22.674	36.309	23.029	18.874
                80	38.894	27.702	22.944	37.539	25.009	21.522
                81	38.217	25.902	21.681	34.549	21.050	15.742
                82	38.847	27.715	22.789	35.301	22.095	18.225
                83	39.199	28.729	23.105	37.491	25.378	21.808
                84	39.074	28.276	23.144	37.572	24.862	21.092
                85	39.212	28.689	23.466	37.610	24.906	21.191
                86	39.141	28.252	22.512	37.865	25.347	21.522
                87	39.380	29.104	23.282	37.925	25.418	21.422
                88	38.499	26.460	21.373	37.996	25.445	21.590
                89	39.113	28.400	22.852	37.067	23.916	19.699
                90	37.882	25.256	20.589	37.184	24.328	20.786
                91	38.305	26.363	22.065	36.781	23.660	19.730
                92	35.884	22.331	16.980	32.802	19.750	13.655
                93	35.559	22.282	18.232	36.495	24.147	20.924
                94	38.148	26.975	22.907	35.167	22.103	18.282
                95	37.322	24.520	20.472	36.870	24.755	20.948
                96	35.069	21.554	16.418	33.672	20.478	14.966
                97	34.965	21.815	17.551	34.371	21.423	17.431
                98	27.276	16.732	8.217	34.124	21.210	17.317
                99	24.589	15.873	7.525	32.307	19.719	14.724
                100	34.660	24.848	22.622	35.800	23.745	20.607
                101	37.836	27.271	22.588	35.452	22.547	19.042
                102	38.520	27.327	22.579	34.830	21.788	17.778
                103	38.624	27.338	22.614	36.401	23.980	20.132
                104	32.308	19.340	11.759	36.844	24.529	20.735
                105	36.020	23.577	20.307	35.018	21.848	17.218
                106	34.948	21.744	17.355	27.767	17.082	8.868
                107	37.890	26.566	22.674	33.892	22.446	19.880
                108	37.456	24.899	20.856	33.265	20.823	16.579
                109	38.001	26.070	21.723	35.874	23.836	20.718
                110	35.516	21.951	16.621	35.950	23.279	19.428
                111	34.654	21.258	16.349	29.870	18.172	10.891
                112	36.130	23.364	19.668	31.445	19.158	15.758
                113	36.053	22.989	19.460	32.181	19.651	16.476
                114	37.017	24.591	21.366	30.939	18.655	14.481
                115	37.442	25.026	21.146	34.385	21.664	19.838
                116	36.588	23.478	19.435	31.078	18.602	13.355
                117	37.363	24.904	21.189	29.898	18.023	13.167
                118	37.048	24.198	20.551	31.418	19.174	15.934
                119	37.322	24.676	20.703	23.261	15.257	7.051
                120	37.176	24.509	20.567	30.446	18.651	18.518
                121	32.729	19.568	12.885	32.560	19.379	18.340
                122	35.363	22.618	19.062	24.947	15.854	7.964
                123	36.584	24.083	20.466	31.674	19.378	18.844
                124	32.320	19.346	12.889	22.337	13.890	6.780
                125	32.800	20.335	15.826	27.215	16.350	14.018
                126	30.597	18.508	12.508	20.596	13.379	5.902
                127	25.911	16.331	8.276	27.996	17.985	18.282
                128	22.559	15.260	6.286	31.642	19.428	18.580
                129	29.536	19.492	16.009	28.592	16.451	12.162
                130	31.586	20.099	16.222	30.736	18.236	17.114
                131	29.370	18.054	12.234	27.686	16.161	12.125
                132	26.829	16.836	9.605	28.269	16.609	13.883
                133	31.148	20.030	16.330	28.405	16.624	13.806
                134	29.563	18.244	12.407	30.609	18.605	17.815
                135	34.545	23.142	20.222	30.883	17.946	15.814
                136	32.354	19.783	14.435	29.189	16.820	13.560
                137	35.627	23.905	20.317	30.973	18.834	17.902
                138	35.941	23.460	19.389	31.505	18.540	17.013
                139	35.766	23.127	19.217	21.484	13.690	6.627
                140	34.557	21.083	16.910	25.876	16.324	15.119
                141	31.677	19.262	13.332	24.084	14.970	10.903
                142	33.347	20.448	17.467	19.628	13.257	6.478
                143	34.314	21.193	18.251	17.306	12.655	5.329
                144	34.733	21.455	18.105	24.042	16.258	16.196
                145	29.173	17.517	10.928	26.710	16.925	15.758
                146	32.145	19.692	16.634	28.047	17.106	15.305
                147	35.074	22.466	20.119	28.420	17.076	15.059
                148	34.736	20.608	18.585	19.810	13.252	5.805
                149	33.640	20.501	16.422	21.489	14.170	9.976
                150	31.708	19.081	14.572	26.661	17.083	16.671

                Comment


                • #9
                  The data is extremely low quality, across the read. And the quality is very spiky; there are a lot of locations where it suddenly plummets. For example, looking at the mhist, at cycle 60 the error rate reaches 8%, and at cycle 98-99 it reaches 15% and 18% respectively. Also, the quality scores are extremely inaccurate, with the error rate indicating accuracy around 14 Phred-scaled points below the claimed quality. Looking at the base frequency histogram, it's also clear that in many cycles the base-caller is erroneously calling Cs as As. Personally, I would not want to use such data; it looks like the run had major problems, or else it is old V1 NextSeq chemistry which was terrible. I would contact the sequencing company and see if they will re-run it for you. That is, if the entire run looks like this; often the first few reads are worse than the rest. For an accurate picture, you should subsample 1 million reads randomly from the files, map those, and use them for your statistics:

                  reformat.sh in1=r1.fq in2=r2.fq out1=s1.fq out2=s2.fq samplereadstarget=1000000

                  Trimming the reads down to ~96bp might make the mapping go a lot faster, as that would get rid of the majority of the worst error spikes. Some of the cycles toward the end have error rates of nearly 30%, which make mapping very slow.
                  Last edited by Brian Bushnell; 11-16-2016, 04:37 PM.

                  Comment


                  • #10
                    Thanks Brian. I will look at the other samples as well and see if they're somewhat better.
                    It seems that those problems are more prominent in the second read of each pair.
                    A rerun is unlikely since the results were posted about 4 months ago. I'll be more allerted for future samples. This sample was analyzed earlier with the hg19 mapping and we have found what we needed. However I used it to see if I can figure out how to do a read mapping by myself because we may get FastQ data only in the future and I may want to switch to hg38 in general.

                    Comment


                    • #11
                      Just to close this thread and give a final answer:
                      The final time for the mapping of a 85 million paired end read human genome on the hg38 reference sequence with BBMap was 520827 seconds = 8680 minutes = 145 hours = just above 6 days.
                      I used 16 cores / 32 threads.
                      The result was as expected.

                      Comment

                      Latest Articles

                      Collapse

                      • seqadmin
                        Strategies for Sequencing Challenging Samples
                        by seqadmin


                        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
                        03-22-2024, 06:39 AM
                      • seqadmin
                        Techniques and Challenges in Conservation Genomics
                        by seqadmin



                        The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                        Avian Conservation
                        Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                        03-08-2024, 10:41 AM

                      ad_right_rmr

                      Collapse

                      News

                      Collapse

                      Topics Statistics Last Post
                      Started by seqadmin, 03-27-2024, 06:37 PM
                      0 responses
                      12 views
                      0 likes
                      Last Post seqadmin  
                      Started by seqadmin, 03-27-2024, 06:07 PM
                      0 responses
                      11 views
                      0 likes
                      Last Post seqadmin  
                      Started by seqadmin, 03-22-2024, 10:03 AM
                      0 responses
                      53 views
                      0 likes
                      Last Post seqadmin  
                      Started by seqadmin, 03-21-2024, 07:32 AM
                      0 responses
                      69 views
                      0 likes
                      Last Post seqadmin  
                      Working...
                      X