One-liner to parse 454BaseCallerMetrics.txt

[pmiguel]

After processing usually follows the maddeningly slow crawl through 454BaseCallerMetrics.txt to find out whether it is a good day or not. Typically I egrep for a few keywords (eg "totalBases") to get an idea. But, even then I am squinting at the screen, counting digits to see if all went well. How about a nice perl one-liner that will pull the lines of interest and add commas in the right places in long numbers?

perl -ne 's/(\d)(?=(\d{3})+(\D|$))/$1\,/g; print if /region|numberReads|totalBases|average|key/;' 454BaseCallerMetrics.txt

That sweet "insert commas into numbers" regex courtesy of Randall Schwartz:

http://www.perlmonks.org/?node_id=28331

--
Phillip

[pmiguel]

By the way, my take on how the regex functions is:

s{
    (\d)            #capture a single digit
    (?=            #so-called look ahead assertion
       (\d{3})+ #3 digits -- remember greediness, though!
       (\D|$)     #a single "not a digit" or end of string (line)
    )
}
{$1\,}gx
#replace the first captured digit with itself plus a comma -- repeat until the end of the string is reached

The goofy "(?=)" syntax requires the regex inside be true, but does not actually "consume" the characters it matches.

One other factor in play: most repeat regex operators are naturally "greedy". That is, they try to match as much as possible. So \d{3}+ will match as many instances as possible of groups of three digits. The result is that the first comma to be replaced is the leftward-most one.

The "g", of course, calls for the matching to be repeated until the end of the string is reached. This is where the (?=) becomes important. Even though it matched all the way to the end of the number, it is a "zero width assertion", meaning string matching picks up again right where the (?=) had started. Hence the next comma can be insert, and so on.

--
Phillip