[pmiguel]

We have implemented the software to demux and convert bcl to fastq off line, which is necessary when the number of samples gets too high. Does anyone know how to toggle a MiSeq between doing all the processing during a typical run (a few small genomes) or generating only bcl files during a highly multiplexed run? Beyond a certain number of samples, the MiSeq chokes after the index reads and it takes hours after the run is complete before the fastq files show up in the run folder in MiSeqOutput. Often it fails to even transfer all the fastq files into the run folder of MiSeqOutput. Has anyone else encountered this problem?

How many indexes before this becomes an issue?

If you changed your sample sheet to only show some of the index pairs maybe that would speed it up. (You would need a full sample sheet for where ever you were running CASAVA off-site.)
--
Phillip

[rnaeye]

Hi!
If you are fine getting index reads as a separate fastq file, Miseq has an option to turn it on. If you turn on relevant flag, it will print fastq files for each indices as well as reads. It's described in MiseqReporterUserGuide. They just need to add following line into appSettings in .exe.config file.

<add key="CreateFastqForIndexReads" value="1" />

Details are written in the pdf manual.

There might be also an option that allows you stop demultiplexing on Miseq. Call Illumina they are very helpful. Only the way I know is .bcl to fastq conversion using a script, on which others have already commented.

[Vinz]

We have implemented the software to demux and convert bcl to fastq off line, which is necessary when the number of samples gets too high. Does anyone know how to toggle a MiSeq between doing all the processing during a typical run (a few small genomes) or generating only bcl files during a highly multiplexed run? Beyond a certain number of samples, the MiSeq chokes after the index reads and it takes hours after the run is complete before the fastq files show up in the run folder in MiSeqOutput. Often it fails to even transfer all the fastq files into the run folder of MiSeqOutput. Has anyone else encountered this problem?

There was a timeout in the MiSeq software for copying the fastq to the output folder. For V3 runs with many samples, copying stopped when exceeding this time. I think this was fixed with 2.4.1. Sometimes copying still fails but most of the times it works in our hands (~1000 to 1500 samples/run).

[drdna]

I don't see why anyone would want to do their own demultiplexing. The demultiplexed data that come off BaseSpace are perfectly good. What are you hoping to accomplish by doing it yourself - salvage the few tens of thousands of reads that can't be recognized?

[GenoMax]

I don't see why anyone would want to do their own demultiplexing. The demultiplexed data that come off BaseSpace are perfectly good. What are you hoping to accomplish by doing it yourself - salvage the few tens of thousands of reads that can't be recognized?

There are regulatory requirements that prohibit use of basespace at some institutions (specially with human samples). When several sequencers are involved, having a common data pipeline is convenient.

[kmcarr]

I don't see why anyone would want to do their own demultiplexing. The demultiplexed data that come off BaseSpace are perfectly good. What are you hoping to accomplish by doing it yourself - salvage the few tens of thousands of reads that can't be recognized?

I do mainly for consistency of final output formats between HiSeq and MiSeq data. Bcl2fastq output file names are:

<SampleName>_<Index>_<Lane>_<Read>_<FileSegment>.fastq.gz

It also includes the actual index read sequence as part of the sequence header.

MiSeq Reporter replaces Index information with "Sample Number" which is nothing more than the order the sample name appeared in the SampleSheet.csv file. A consequence of this is that if you happen to sequence exactly the same sample on two different runs, but the samples were listed in a different order in the SampleSheet then its output name will be different between the two runs.

The "Sample Number" (order in SampleSheet.csv) is a meaningless/worthless piece of information made even more meaningless and worthless by the fact that it is not necessarily consistent from run to run. It was pretty stupid on Illumina's part to enshrine it the file names and headers, and pretty annoying on Illumina's part that they create different naming conventions for the two platforms even though the data is identical.

[Reid]

Stop automatic demultiplexing on MiSeq

[QUOTE=drdna;144293]I don't see why anyone would want to do their own demultiplexing.

Additionally, my pre-demultiplexed sequences are presenting challenges for downstream primer sequence removal using mothur. The program has workflows for removing primers along with barcodes and indices, but not independently.