SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Live Webinar on UMI-powered Ultra-sensitive Variant Calling on 13 Dec 2017 Strandlife Bioinformatics 2 12-12-2017 10:43 PM
Help with UMI (unique molecular identifiers) data processing donquijotes Bioinformatics 12 12-01-2017 11:59 PM
UMI in Smart-seq2 protocol PrimerBuffalo Sample Prep / Library Generation 1 05-11-2017 04:10 PM
FASTQ manipulation in Galaxy blankenberg Bioinformatics 6 11-21-2012 12:59 AM

Reply
 
Thread Tools
Old 12-13-2017, 04:49 AM   #1
dimo
Member
 
Location: Australia

Join Date: Mar 2012
Posts: 10
Default Fastq manipulation (UMI)

Greetings,

I have some targeted DNA seq data with UMIs, where the UMI barcode is part of the read header as such:

NB500916:322:HVV53AFXX:1:11101:8946:1524 1:N:0:TGAAGAGA+AATGCTCCGT
CAGGGTGGAAAAGGGGTCCTGGGCTTCAGCTGAAGGGCAAACTGCCCAGTGTAGGAGTCCGTCCAGGACAGGCAG

Where TGAAGAGA is the index and AATGCTCCGT is the UMI id.

To run through various UMI pipelines I need to have the UMI is as part of the read ID (to use with UMI-tools) or as the actual sequence (to use with fgbio).

So the two outputs I need are (highlighting the changes):

1)

NB500916:322:HVV53AFXX:1:11101:8946:1524:AATGCTCCGT 1:N:0:TGAAGAGA+AATGCTCCGT
CAGGGTGGAAAAGGGGTCCTGGGCTTCAGCTGAAGGGCAAACTGCCCAGTGTAGGAGTCCGTCCAGGACAGGCAG

2)

NB500916:322:HVV53AFXX:1:11101:8946:1524 1:N:0:TGAAGAGA+AATGCTCCGT
AATGCTCCGT

Any help would be greatly appreciated.
dimo is offline   Reply With Quote
Old 12-13-2017, 03:08 PM   #2
neavemj
Member
 
Location: MA, USA

Join Date: Feb 2014
Posts: 58
Default

Hi dimo,

Are you analyzing this data on a linux machine? The following sed command should do what you want for the first case:

sed "s/\(NB.*\)\(\s.*+\)\(.*\)/\1:\3\2\3/g" original_file > new_file

This relies on matching 3 patterns in your header lines; 1) 'NB' up until the first space, 2) the space until the first + symbol, and 3) anything following the + symbol (i.e. your barcode). Once these 3 patterns are matched, the next part (\1:\3\2\3) rearranges the patterns in the order that you want.

For case 2, do you want just the barcode on the next line, or should the barcode be in front of the actual sequence?

Cheers,

Matt.
neavemj is offline   Reply With Quote
Old 12-13-2017, 03:14 PM   #3
dimo
Member
 
Location: Australia

Join Date: Mar 2012
Posts: 10
Default

Quote:
Originally Posted by neavemj View Post
Hi dimo,

Are you analyzing this data on a linux machine? The following sed command should do what you want for the first case:

sed "s/\(NB.*\)\(\s.*+\)\(.*\)/\1:\3\2\3/g" original_file > new_file

This relies on matching 3 patterns in your header lines; 1) 'NB' up until the first space, 2) the space until the first + symbol, and 3) anything following the + symbol (i.e. your barcode). Once these 3 patterns are matched, the next part (\1:\3\2\3) rearranges the patterns in the order that you want.

For case 2, do you want just the barcode on the next line, or should the barcode be in front of the actual sequence?

Cheers,

Matt.
Hi Matt, thanks for the help. Using a a linux machine, and the sed code works a treat. For case two I just need the barcode on the next line, as AnnotateBamWithUmis will just match bam and fast based on read ID and add sequence to the RX annotation of the bam. Which I can then use downstream.

Thanks again, really appreciate it.
dimo is offline   Reply With Quote
Old 12-13-2017, 03:31 PM   #4
neavemj
Member
 
Location: MA, USA

Join Date: Feb 2014
Posts: 58
Default

In that case, you can rearrange that sed command like so:

sed "s/\(NB.*\)\(\s.*+\)\(.*\)/\1\2\3\n\3/g" original_file > new_file

This is now saying 'insert a newline character (\n) before writing the barcode'. You probably don't need all that pattern matching in this case but since it's already written you may as well leave it.

So this will slip the barcode in just before the sequence, which will be written on the next line. If you don't want the sequence in the file at all, I would probably grep the headers out, then pipe them to the sed command:

grep "NB" original_file | sed "s/\(NB.*\)\(\s.*+\)\(.*\)/\1\2\3\n\3/g" > new_file

Cheers,

Matt.
neavemj is offline   Reply With Quote
Old 09-26-2018, 11:38 AM   #5
CarnifexRex
Junior Member
 
Location: Dallas

Join Date: Sep 2018
Posts: 4
Default

I know this was answered almost a year ago, but I'm interested in case 2, except that I would want to keep the original sequence, just put the umi sequence infront.

So:

NB500916:322:HVV53AFXX:1:11101:8946:1524 1:N:0:TGAAGAGA+AATGCTCCGT
AATGCTCCGTCAGGGTGGAAAAGGGGTCCTGGGCTTCAGCTGAAGGGCAAACTGCCCAGTGTAGGAGTCCGTCCAGGACAGGCAG

I got as far as creating the new line with the umi sequence, but then the read sequence ends up in a third line

Also, my data would be FASTQ, so I'm guessing I'd also need to add some bogus quality scores? How can I add characters to every 4th line?
CarnifexRex is offline   Reply With Quote
Old 09-30-2018, 09:28 PM   #6
neavemj
Member
 
Location: MA, USA

Join Date: Feb 2014
Posts: 58
Default

Hi CarnifexRex!

That's getting a bit trickier because you want to do something with the 'next' line as well as the matched line. Here is how I would do it, although I think there are probably cleaner ways:

sed "/^NB/{N;s/\n/\t/}" original_file | sed "s/\(^NB.*+\)\(.*\)\t\(.*\)/\1\2\n\2\3/g" > new_file

In this case, I'm using sed to grab the line starting with "NB", then also grabbing the next line, then replacing the newline character with a tab. This means I can then pipe this whole thing (which contains all the information we need) to another sed command. Here, pattern 1 is the "NB" up until the plus symbol, pattern 2 is the UMI, and pattern 3 is the sequence. The next bit rearranges these patterns as you wanted.

If you want to make it look like a fastq file, you can continue adding characters to the second sed command. To simply add dummy "?" symbols for the quality, you could do this:

sed "/^NB/{N;s/\n/\t/}" original_file | sed "s/\(^NB.*+\)\(.*\)\t\(.*\)/\1\2\n\2\3\n+\n$(printf '?%.0s' {0..84})/g"

Just double check that the output looks right, etc. I wrote this fairly quick and probably didn't test it enough! If there are any unexpected tab or newline characters in the file it won't work .

Let me know how you go!

Cheers,

Matt.
neavemj is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 03:41 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO