SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Sam file smaller than fastq scami Bioinformatics 6 10-01-2015 05:25 AM
Read in mutiple gzipped FastQ files using R Nicolas_15 Bioinformatics 4 09-04-2015 01:47 PM
fastx quality trimmer and gzipped fastq balsampoplar Bioinformatics 4 03-10-2014 06:53 AM
Script for breaking large .fa files into smaller files of [N] sequences lac302 Bioinformatics 3 02-21-2014 04:49 PM
Split fastq into smaller files lorendarith Bioinformatics 10 12-13-2012 04:28 AM

Reply
 
Thread Tools
Old 04-11-2019, 03:01 AM   #101
Chief_Lazy_Bison
Junior Member
 
Location: Ames, IA

Join Date: Dec 2014
Posts: 9
Default

So I resubmitted the job on a node with 40 processors and 1TB of memory and I received two very similar exceptions and the job is hanging again.

Exception in thread "Thread-147" java.lang.AssertionError
at clump.KmerSort3$FetchThread3.fetchNext_inner(KmerSort3.java:706)
at clump.KmerSort3$FetchThread3.fetchNext(KmerSort3.java:655)
at clump.KmerSort3$FetchThread3.run(KmerSort3.java:577)
--
Exception in thread "Thread-146" java.lang.AssertionError
at clump.KmerSort3$FetchThread3.fetchNext_inner(KmerSort3.java:706)
at clump.KmerSort3$FetchThread3.fetchNext(KmerSort3.java:655)
at clump.KmerSort3$FetchThread3.run(KmerSort3.java:577)
Chief_Lazy_Bison is offline   Reply With Quote
Old 04-11-2019, 03:53 AM   #102
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,966
Default

Can you provide the exact command line you are using? Is this being submitted via a job scheduler?
GenoMax is offline   Reply With Quote
Old 04-11-2019, 04:24 AM   #103
Chief_Lazy_Bison
Junior Member
 
Location: Ames, IA

Join Date: Dec 2014
Posts: 9
Default

It is submitted to a SLURM queue via the attached script.

These reads are a collection of concatenated interleaved paired end libraries

The same script worked well on the individual libraries, but I wanted to do an assembly with all of the reads together so I concatenated them all with
Code:
cat *fq.gz > ALL.fq.gz
The command that ends up stalling is this:
Code:
clumpify.sh in=ALL_temp.fq.gz out=ALL.eccc.fq.gz ecc passes=4 reorder

bbmerge plows through these reads with no complaints just prior to clumpify

Code:
bbmerge.sh in=ALL_temp.fq.gz out=ALL.ecco.fq.gz ecco mix vstrict ordered ihist=ALL_ihist_merge1.txt
Attached Files
File Type: txt ALL_ec.SLURM.txt (6.5 KB, 1 views)
Chief_Lazy_Bison is offline   Reply With Quote
Old 04-11-2019, 06:12 AM   #104
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,966
Default

I think you should follow the order of tools that Brian has in his script example. Do clumpify job first. Since you are merging the reads first I am going to speculate that clumpify is unable to identify duplicates properly. If your data in not from a patterned flowcell you could remove the "optical" flag for clumpify.
GenoMax is offline   Reply With Quote
Old 04-15-2019, 03:17 AM   #105
Chief_Lazy_Bison
Junior Member
 
Location: Ames, IA

Join Date: Dec 2014
Posts: 9
Default

Thank you for the quick advice. I had attempted to merge many samples together at the front end of the pipeline so that I could to all the QC and error correction at once. My problem was fixed when I did QC and error correction on each sample individually and then merged for a co-assembly.

Thanks again.
Chief_Lazy_Bison is offline   Reply With Quote
Old 05-23-2019, 07:15 AM   #106
DCZ
Junior Member
 
Location: Europe

Join Date: Feb 2019
Posts: 4
Default

Hi all,

I was wondering why the default for spantiles is set to false. If a read for instance has coordinates (1000,1000) and the dupedist is set to 2500, (see sketch attached), there's a possible overlap with 3 other tiles. So even if it's not a NextSeq, but a HiSeq4000 for instance, there are no tile-edge duplicates, however there's still a possibility that optical duplicates end up on neighboring tiles (or even further). Can anyone elucidate on this?

Thanks in advance!

Attachment: The dot represents the "original read", the circle represents the distance of 2500 around the "original read". Rectangles represent tiles.
Attached Images
File Type: png Screenshot from 2019-05-23 17-11-08.png (4.3 KB, 2 views)

Last edited by DCZ; 05-23-2019 at 07:27 AM.
DCZ is offline   Reply With Quote
Old 05-24-2019, 09:57 AM   #107
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,966
Default

Illumina's software pre-processing takes care of clusters that may be showing mixed signals etc so they may never pass that step. Spantiles=t is mainly for nextSeq, where the clusters are hugh (relatively) and as a result there is a chance they will cross tiles. I believe this was done based on empirical observation Brian had done when he was developing clumpify.
GenoMax is offline   Reply With Quote
Old 05-26-2019, 11:55 PM   #108
DCZ
Junior Member
 
Location: Europe

Join Date: Feb 2019
Posts: 4
Default

Thanks for your reply. I'm still confused though. Just like there can be empty wells on the same tile, there can also be empty wells on neighboring tiles (correct me if i'm wrong). I suppose these wells would not show a mixed signal but would just get filled with a duplicate in the same way as the optical duplicates get formed on the same tile.
DCZ is offline   Reply With Quote
Reply

Tags
bbduk, bbmap, bbmerge, clumpify, compression, pigz, reformat, tadpole

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 02:13 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO