SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Parser For Illumina InterOp Binary Metric Files iamh2o Bioinformatics 17 09-06-2016 06:40 AM
Illumina InterOp Parsers (perl and R) mchen1 Bioinformatics 18 02-19-2016 11:27 AM
Metrix - A Server / client parser for Illumina (InterOp) run directories Rhizosis Bioinformatics 29 10-15-2014 05:06 AM
Ruffus - a lightweight python module to run computational pipelines Chrn Bioinformatics 2 09-01-2010 08:31 AM
can i run single application process on different processor er.surendersharma Bioinformatics 6 05-31-2010 10:09 PM

Reply
 
Thread Tools
Old 07-24-2013, 05:49 PM   #1
nthmost
Junior Member
 
Location: San Francisco, CA

Join Date: Mar 2013
Posts: 3
Default Illuminate: processor for Illumina InterOP Run binaries (in Python)

Greetings all,

I work at InVitae and we just publicly released a library called Illuminate.

https://bitbucket.org/invitae/illuminate/

The purpose of Illuminate is to emulate the stats you see when you load a run data folder within Illumina SAV, providing programmatic access to these metrics for whatever purposes you may have -- data storage, analysis, automated machine monitoring, and so on.

This is completely free, open source software (MIT License) written in Python with the intent to be used, tested, and improved upon by the bioinformatics community.

Features:
  • Simple command-line tool you can use to quickly inspect a run.
  • Built to be easily integrated into other code.
  • Easily extensible even if you think you are "not much of a programmer".
  • Results standardized to pandas DataFrame objects (so if you know how to work in R, you can probably get up to speed quickly with this)

Here's an example of the smallest python script you could get away with using this tool.

Code:
import illuminate
myDataset = illuminate.InteropDataset('path/to/rundata/')
print myDataset.meta
print myDataset.IndexMetrics()
print myDataset.TileMetrics()
print myDataset.QualityMetrics()
And here's an example of how you would use the command-line reporter to do the same thing:

Code:
python illuminate --meta --index --tile --quality /path/to/rundata
You can even have illuminate open up in an interactive iPython shell, where the dataset will be loaded up into an InteropDataset object for you:

Code:
python illuminate -i /path/to/rundata
Not all of the metrics objects are fully fleshed out yet, although all of the binary parsers are "feature complete" in that you can produce a data dictionary and a DataFrame from them.

I'm hoping that some of you fine folks can pipe up and let me know what might be useful to you -- or better, submit contributions, bug reports, and so on that will help Illuminate become as full-featured as it needs to be.

This library has been in our production pipeline for several months now, reporting on cluster density, quality, and yield so we can keep tabs on sequencing run quality in an automated fashion.

If you use it, or you have questions about it, please comment here and let me know!

Cheers,
Naomi
nthmost is offline   Reply With Quote
Old 07-25-2013, 09:59 AM   #2
lingfung.tang
Member
 
Location: San Francisco

Join Date: Dec 2010
Posts: 10
Default

interesting, never thought of cracking the bin files.. thanks.
lingfung.tang is offline   Reply With Quote
Old 07-31-2013, 05:16 PM   #3
nthmost
Junior Member
 
Location: San Francisco, CA

Join Date: Mar 2013
Posts: 3
Default Now available through pypi (pip)

I just pushed the latest stable version of Illuminate to the public pypi server, so you can install it via:
  • pip install illuminate

Note that, due to weirdness in python packaging, you will probably have to install two of its dependencies manually:
  • pip install numpy
  • pip install pandas

More detailed instructions can be found on the pypi.python.org page for Illuminate.

Stable versions will always be found there; development versions will always come from the bitbucket repository (see original post in this thread).

Cheers,
Naomi
nthmost is offline   Reply With Quote
Reply

Tags
hiseq, illumina, miseq reporter, open source, python

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 04:46 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO