Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • One question to the new published paper about Myrna

    The latest paper about RNA-seq application is "Cloud-scale RNA-sequencing differential expression analysis with Myrna" written by Ben Langmead, etc. It uses Bowtie, R, Bioconductor. I think they are integrated with Hadoop streaming, since they are all binary executable tools. My question is how to install them on all of hadoop nodes in one step or quickly? Manually installation will be a tedious work and not acceptable when you have a big hadoop cluster, say, over 100+ nodes. I just wonder is there a solution to overcome or fix this issue.
    Thanks in advance.
    BTW, I don't think binary executeables installation in a shared file system like NFS is a good solution.
    Last edited by xinwu; 08-22-2010, 07:38 PM.

  • #2
    Originally posted by xinwu View Post
    BTW, I don't think binary executeables installation in a shared file system like NFS is a good solution.
    Please clarify the above.

    Comment


    • #3
      Originally posted by nilshomer View Post
      Please clarify the above.
      OK, I should state that "not a good solution for me" rather than "not a good solution" because of some internal limitations.
      But NFS itself has limitations if you considerate performance, etc.

      Comment


      • #4
        Originally posted by xinwu View Post
        Manually installation will be a tedious work and not acceptable when you have a big hadoop cluster, say, over 100+ nodes. I just wonder is there a solution to overcome or fix this issue.
        Thanks in advance.
        BTW, I don't think binary executeables installation in a shared file system like NFS is a good solution.
        Hi Xinwu,

        I'm not sure why a shared filesystem isn't a good solution -- it works for me, and is way more convenient than installing on each node. As for easy installation: we include a script with Myrna that automatically grabs and builds R and Bioconductor with all of the appropriate packages installed ($MYRNA_HOME/R/build_r.sh). The ideal situation is: you download the Myrna archive to an NFS share, run that script from one of your cluster nodes, and you're done. R is now runnable on all the nodes, and Myrna automatically tries $MYRNA_HOME/R when looking for an appropriate R_HOME.

        Also note that if you run in cloud mode (on EMR), R is downloaded to each EC2 node automatically with no user intervention.

        Thanks,
        Ben

        Comment


        • #5
          Hi Ben,
          First, I really appreciate your excellent job on NGS. Actually, I met a potential NFS bug in my computing environment and can not build/install any binary software. I am also not authorized to update Linux/NFS to fix the issue. I apologize if I mislead you by "not a good solution", what I want to say is "is not a choice for me". In my situation, I prefer to install myrna on native system rather than NFS due to the issue I met and some concern of performance, security policy, etc. You said when myrna is run in cloud mode(EMR), R is installed on EACH EC2 node. It indicates cloud mode also does not use shared NFS. The package is actually installed on the native system of every node, right? How do you do that? Or it is done automatically by Amozon Infrastructure? Thanks.

          Comment

          Latest Articles

          Collapse

          • seqadmin
            Strategies for Sequencing Challenging Samples
            by seqadmin


            Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
            03-22-2024, 06:39 AM
          • seqadmin
            Techniques and Challenges in Conservation Genomics
            by seqadmin



            The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

            Avian Conservation
            Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
            03-08-2024, 10:41 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, Yesterday, 06:37 PM
          0 responses
          8 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, Yesterday, 06:07 PM
          0 responses
          8 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 03-22-2024, 10:03 AM
          0 responses
          49 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 03-21-2024, 07:32 AM
          0 responses
          66 views
          0 likes
          Last Post seqadmin  
          Working...
          X