parser needed for bcl2fastq2 output

[lac302]

I'm looking for a parser to compile basic stats from the various html/xml/text output files from NextSeq runs. Looking for a text file to archive with fastq files that includes sample name, sample ID, indexes, total clusters per sample not per lane since NextSeq lanes are not independent. Thanks.

[GenoMax]

Take a look at MultiQC if you want to look at many runs.

Following is true for local bcl2fastq analysis. I am not familiar with BaseSpace but I suppose a similar structure can be found there as well.

Otherwise the index.html file found in (FCID/Unaligned/Reports/html) directory has all the stats or if you prefer a standalone file then (FCID/Unaligned/Reports/html/FC_BARCODE/all/all/all/laneBarcode.html).

JSON format results are in (FCID/Unaligned/Stats/Json.stats).

[lac302]

Thanks GenoMax. We are running our NextSeq standalone.

The html file displays the metrics per lane per sample...I'd like to have the information for each sample combined into one row, e.g. Lane 1 Sample A, Lane 2 Sample A, Lane 3 Sample A, Lane 4 Sample A -> Sample A.

If you know of an html to csv or json to csv script I could handle this in Excel.

[GenoMax]

While there appear to be many (?) online tools following may be safer.

1. Download Atom programmers editor here.
2. Find "Settings" tab after installing Atom and click on +Install.
3. Search for a package called "json-converter" and install it.
4. Download and open Stats.json file in Atom.
5. Use Packages menu drop down, find "Json Converter" and select "Json to csv".
6. Write the converted data out to file and then do what you need to in Excel.

[moatman]

As a side-note, when running bcl2fastq you can use the option "--no-lane-splitting" to create fastqs that don't separate your samples by lane. This is useful to avoid having to combine them downstream.

[kmcarr]

I'm looking for a parser to compile basic stats from the various html/xml/text output files from NextSeq runs. Looking for a text file to archive with fastq files that includes sample name, sample ID, indexes, total clusters per sample not per lane since NextSeq lanes are not independent. Thanks.

Hello lac302,

I have attached a perl script I use for this purpose, however when dealing with NextSeq run data the stats are still divided by lane. The script reads the DemultiplexingStats.xml and ConversionStats.xml files within the Stats/ directory created by bcl2fastq2. It requires Perl Modules Getopt::Long and XML::LibXML. It has one mandatory input, the path to the Stats/ directory and one optional argument for parsing data from single end runs.

# parseBcl2FastqStatsXml.pl [-s] -i <path>/Stats/ > output.txt

The argument for -i must end in /Stats/
-s is optional for single read runs

The output is a tab delimited text file with relevant per sample, per lane stats.

[lac302]

Guess what?...You can copy and paste from html to excel.

I was making it harder than it needed to be.

[jgarces]

@kmcarr, thanks for that useful parser! One quick question: is there any way to sum each line instead of showing them separately?

Thanks!

[kmcarr]

@kmcarr, thanks for that useful parser! One quick question: is there any way to sum each line instead of showing them separately?

Thanks!

By "sum each line" do you mean the total output per lane? If so then that information is provided directly in the HiSeq output summary before running BCL2FastqQ.

[jgarces]

Thanks for you reply, @kmcarr. Our sequencing were performed in a different lab and we haven't the BCL and this kind of run information... but at the end, with your parser and a bit of R, I was able to extract the information I wanted.

Thanks again!