View Single Post
Old 05-06-2017, 07:19 AM   #74
chiayi
Member
 
Location: New York

Join Date: Dec 2016
Posts: 22
Default

I tried to tune several places and here's a summary of what I found:
1.
Code:
Executing clump.Clumpify [-Xmx16g, in=in.fastq.gz, out=out.fq.gz, dedupe, reorder]
[dedupe reorder][to reproduce Brian's results] In the runs for my previous post, -Xmx was ~80% of the physical memory. With that setting, Oracle JDK (88.88k reads/sec) is ~10 tims faster than Open JDK (9.80k reads/sec). However, when I adjusted -Xmx to 50% of physical memory, the speed increased ~10 times for Oracle JDK (1222.36k reads/sec) and ~120 times for Open JDK (1200.63k reads/sec). Adding reorder and/or addcount didn't make much difference; -Xmx to physical memory ratio is the key.
2.
[dedupe reorder optical dupedist=40][setting in my original post] Then I added back optical and dupedist tags (with -Xmx at 50% of the physical meory). The run was stuck at dedupting like before.
Code:
   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 97955 cc5544    20   0 25.117g 9.998g  12388 S 100.0  4.0  36:38.45 java
 98052 cc5544    20   0 1924368  17736    700 S   0.0  0.0   1:30.90 pigz
Brian, could you try adding these two tags and see if this is reproducible at your end? Thanks a lot!
chiayi is offline   Reply With Quote