Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Recover only longest version of sequence from multiple sequence fasta file - help

    Greetings,
    I have a large list of fasta sequences (not paired end) from which I want to isolate the longest version of each "sequence" (obviously the longest one has a different sequence). For example:

    input.fa:
    >Seq_1
    AAACCCGGGTTT

    Seq_2
    CCCGGG

    output_keep.fa:
    >Seq_1
    AAACCCGGGTTT

    output_ditch.fa:
    >Seq_2
    CCCGGG

    I tried a number of approaches using a mock dataset containing mismatches, exact copies, and known truncations from either end of the long form - "test_dups.fa":

    (1) dedupe.sh in SAMtools
    dedupe.sh test_dups.fa test_dups_dedupe.fa outd=test_dups_removed.fa

    this didn't work: its output didn't make sense, and an exact duplicate pair was removed completely (didn't leave behind one of them, which is bad news).

    (2) with the ac=f option
    dedupe.sh test_dups.fa test_dups_dedupe.fa outd=test_dups_removed.fa ac=f

    that only removed the exact copy pair, again not leaving one copy in file.

    (3) clustering by overlap
    dedupe.sh test_dups.fa test_dups_dedupe.fa outd=test_dups_removed.fa ac=f mo=50 c pc csf=stats.txt outbest=test_best.fa mcs=1

    didn't work (retained were not longest examples).

    Then I moved on to VSEARCH, to sort by size, then form clusters and output the "seed":

    (4) vsearch clustering
    --cluster_fast dups.fa --centroids test_unique_1.0.fa --id 1.0
    oddly, this kept a truncation of the longest sequence, but was close...

    (5) using the iddef = 0, to ignore end gaps
    vsearch --cluster_fast dups.fa --centroids test_unique_1.0.fa --id 1.0 --iddef 0

    this did not ignore the end gaps penalties as expected.

    (6) Align two seqs and see what the perceived idenity is
    vsearch --allpairs_global dups_9_10.fa --acceptall --alnout test_aln.fa --iddef 0

    the resulting alignment shows 100% identity, as expected with the iddef 0 option.

    Why are those sequences not in the same cluster then?

    Searching for help on "remove duplicates" is a disaster here, so I hope I can get some help. One consideration is to merge the sequences with 100% identity for the "overlap", but those tools merge from one end of the read, as is used to combine paired end reads. The other consideration is spending a day writing a script. I reasoned I can not be the first person to do this.

  • #2
    What I will do is to write my own script to
    1) blastn the sequences against itself (hopefully your sequences are long enough to justify using blast)
    2) filter the results to remove blastn results of the same sequences and min e-value
    3) Do single linkage clustering based on the blastn results
    4) Choose the longest sequence for each cluster

    There should be an easier way by using k-mer?!

    Comment


    • #3
      have you tried CD-hit?

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Strategies for Sequencing Challenging Samples
        by seqadmin


        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
        03-22-2024, 06:39 AM
      • seqadmin
        Techniques and Challenges in Conservation Genomics
        by seqadmin



        The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

        Avian Conservation
        Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
        03-08-2024, 10:41 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, Yesterday, 06:37 PM
      0 responses
      10 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, Yesterday, 06:07 PM
      0 responses
      9 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-22-2024, 10:03 AM
      0 responses
      51 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-21-2024, 07:32 AM
      0 responses
      67 views
      0 likes
      Last Post seqadmin  
      Working...
      X