[Vinz]

We've been using that same format, with the 4 N bases at the beginning, for some runs and have had some success. Success in this case though is ~2 millions usable reads that aren't phiX before you even get into quality trimming and processing. So far we've been able to squeeze out enough data to make it work, and it's more cost effective than the 454, but obviously this is a non-optimal solution for an instrument that's stated to give ~15 million read pairs.

Matrix is calculated based on the first 4 bases. However phasing is calculated based on the first 12 bases. Have you (or someone else) tried to include 12 Ns in the primers? That would be painful to sequence 12 additional bases with best quality of nonsense. But at least you should get around without spiking 75% phiX.

[pmiguel]

Why are phasing/pre-phasing calculations affected by low diversity? I can't think of a mechanism. Unless it is just follow-on issues from bad matrix calculations.

--
Phillip

[mcnelson.phd]

Why are phasing/pre-phasing calculations affected by low diversity? I can't think of a mechanism. Unless it is just follow-on issues from bad matrix calculations. -- Phillip

I can't speak for sure on the mechanism since I don't know all of the details of how it's calculated, but the RTA software expects an even base distribution as part of its phasing/pre-phasing calculations. Phasing/pre-phasing aren't calculated for every molecule and instead a global value is determined that is applied to everything. With low diversity amplicons, from what I understand, the extreme swings in base composition (>50% A in cycle N to <15% A in cycle N+1) really screw up how those values are calculated and thus globally applied. Hence the recommendation to use lots of phiX, which will hopefully be used more than your amplicons to determine the phasing/pre-phasing values.

I'm sure someone with more knowledge than myself can give a better explanation of that.

[Vinz]

Why are phasing/pre-phasing calculations affected by low diversity? I can't think of a mechanism. Unless it is just follow-on issues from bad matrix calculations.

--
Phillip

That's just what I learned in a webinar. I have no idea why the phasing values suffer... However, 4 Ns don't seem to be sufficient if I understand mcnelson correctly.

[GenoMax]

Instructions we just received for using a hard coded matrix and phasing appear to be different than the ones listed here. We are going to try them out today.

BTW: What version of MCS are you running?

Instructions for hardcoded phasing/matrix with upgraded MiSeqs:
Choose a good run (phiX).
Run folder\Data\Intensities\Basecalls\Matrix\s_1_1_matrix.txt
Run folder\Data\Intensities\Basecalls\Phasing\s_1_1_phasing.txt
Copy these 2 files and rename them as hardcodedmatrix.txt and hardcodedphasing.txt respectively.
Place them in C:\Illumina\RTA

[Vinz]

Instructions we just received for using a hard coded matrix and phasing appear to be different than the ones listed here. We are going to try them out today.

BTW: What version of MCS are you running?

We run MCS 2.0.5.0

[LVAndrews]

I'm wondering about hardcoded phasing/matrix files...

In the blog post here (http://pathogenomics.bham.ac.uk/blog...llumina-miseq/), it is indicated that after 10/31, you don't monkey with the .xml file, just save your "nice" phasing and matrix files as phasing.txt and matrix.txt into the RTA directory. I finally ran some 16S runs in January, and I'm not convinced the instrument fell back to the desired values as I had exceptionally high phasing/prephasing. The first run gave the following phasing/prephasing:

300cycles, 50% PhiX, read1: 0.104/0.533, read2: 0.298/0.610

There was a massive intensity spike during read2 (and the qscores not so good), so our FAS sent us a replacement kit. I reran it, but decided to try the "custom read" primer wells, and so I don't know the phasing/prephasing values are trustworthy. But despite losing my PhiX-based metrics by doing this, I assume the complexity was still observed, so they can't be too far off. Also, this was run the day of or the day after the most recent software upgrade, and I realized after the fact the upgrade had removed my phasing/matrix files (as well as my preference to generate an index read fastq):

300 cycles, 25% PhiX, read1: 0.581/0.234, read2: 1.146/0.000

So that run also looked terrible and our FAS sent us another kit to try. The data for read1 look much better this time with most reads at or above q30 at 90% read length, but the phasing/prephasing still look bad. Also, quality took a serious dump on read2:

300 cycles, 25% PhiX, read1: 0.094/0.573, read2: 0.380/1.160

Our genomic runs all look great, so I know this has everything to do with low complexity. Can anyone confirm that with the latest software upgrade we just drop new phasing.txt and matrix.txt into the RTA directory to correct this problem? Any other manipulations of the software that people have found to help?

[Vinz]

I was explained that the difference between modifying the MiSeqConfiguration.xml file and adding the matrix/phasing files is the following:
1. MiSeqConfiguration.xml set hardcoded values will always be used (Nicks post is still up to date for the current release)
2. matrix/phasing txt files will be used in case the software assumes that something with the calculation of matrix/phasing is wrong. Meaning: In extreme cases both methods work the same. However, with version 2 you may run in the situation, that you have used amplicon but RTA has decided to use the calculated phasing/matrix values. That may not give you good results. You know after the run: If the displayed phasing values are those that you have set, it was hardcoded. Otherwise it was not hardcoded.
We prefer version 1 for amplicon sequencing. That seems to work pretty well.

[GenoMax]

If you do not want to mess with the MiSeqConfiguration.xml file each time (since that requires a restart of MCS) you could leave the hard-coded file in place. It does not seem to hurt "normal" runs.

[Vinz]

I do not think a restart of MCS is required to apply a new MiSeqConfiguration.xml file. My understanding is that it will look for the xml at the beginning of the run.
However, if you want to do a run without hardcoding, you would have to change it back... That's definitely true.

[LVAndrews]

Thanks for the advice.

Just to confirm, you can still name the new files "hardcodedphasing.txt" as long as the xml is telling the software what to look for? Or should I plan to just rename files and restart MCS prior to each run if I am switching from high to low complexity templates?

Also, since our first run had phasing and matrix files in place but the phasing/prephasing didn't match the PhiX values, and PhiX was a bit high (50%), it seems to me that if you plan to use hardcoded values then lower PhiX is warranted.

Finally, if I want to keep the image files to toy with different settings (<CopyImages>true</CopyImages>), does anyone have an idea how much space this will require from a single 300 cycle run? Our Miseq only came with a 500GB drive and I don't want it choking half-way through a run. I did network it (gigabit connection) to our local RAID device which has much more space, so I could place the output there, I suppose, but sometimes the network can screw you as well.

[GenoMax]

I do not think a restart of MCS is required to apply a new MiSeqConfiguration.xml file. My understanding is that it will look for the xml at the beginning of the run.
However, if you want to do a run without hardcoding, you would have to change it back... That's definitely true.

We were told to re-launch MCS if XML file was changed. Even if it is not needed it only adds a few minutes as the machine re-initializes.

[Vinz]

If you want to follow option 2, then you do not have to change the xml. Then lower PhiX may give you better result as high PhiX might prevent hardcoded phasing and still have bad values. If you want to follow option 1, then change the xml according to Nicks post. If you save a xml version with and without hardcoding, you may switch between those by renaming the files.

[GenoMax]

Thanks for the advice.

Just to confirm, you can still name the new files "hardcodedphasing.txt" as long as the xml is telling the software what to look for? Or should I plan to just rename files and restart MCS prior to each run if I am switching from high to low complexity templates?

You can keep two copies of MiSeqConfiguration.xml file. One for the "normal" runs and the other for "low complexity". Make sure you move the correct file in place before starting the run and rename the other so there is only one "MiSeqConfiguration.xml" file available.

Finally, if I want to keep the image files to toy with different settings (<CopyImages>true</CopyImages>), does anyone have an idea how much space this will require from a single 300 cycle run? Our Miseq only came with a 500GB drive and I don't want it choking half-way through a run. I did network it (gigabit connection) to our local RAID device which has much more space, so I could place the output there, I suppose, but sometimes the network can screw you as well.

We routinely use NAS for data capture and have never had problems (unless your local network is not reliable).

[pmiguel]

Anybody using 500 cycle kits for amplicons sequencing? Really need to make that work to completely supplant GS-FLX amplicon reads.

Also, it was my understanding that the new MCS version 2.1.1.13 had some improvements for handling low diversity data. Anyone seen evidence of this?

--
Phillip