Bowtie2 shared memory (--mm)

[mscholz]

Anyone knows how this bad boy works? The manual is less than informative.

How do different threads know the shared memory is there? What's the deal???

Help?

[nilshomer]

Originally Posted by mscholz

Anyone knows how this bad boy works? The manual is less than informative.

How do different threads know the shared memory is there? What's the deal???

Help?

I would presume that the main thread memory maps the index file so that the index is treated as though it is in primary memory. Then the main thread needs only give a pointer to this memory to each thread. This is no different than if it had just read the index into memory. That's the beauty of memory mapping, as it is as though the file is in memory from the point of view of the process.

For shared memory, typically a key is used to associate a shared memory segment (google shmget for example). Given this key, a process can attach the shared memory to its own process memory.

[tonybolger]

Originally Posted by mscholz

Anyone knows how this bad boy works? The manual is less than informative.

How do different threads know the shared memory is there? What's the deal???

Help?

It's magic

Seriously though, it's probably just memory mapping the index files (via mmap). The OS can automatically share multiple mappings to the same file from different processes, so it allocates RAM for only one copy in memory.

BTW, threads within one bowtie process automatically share their memory, this only saves memory when running multiple bowtie processes.

*edit: beaten to it

[mscholz]

So,

I'm back to this. My question is this, how on earth would I call bowtie2 using multiple read sets on the -mm flagged index so that I don't have to keep reloading the same index? Do I have to run it with parallel? I'm just not clear at all on how you get two disparate calls of the same index to share the memory mapping info...

Thanks!

[arolfe]

You add the --mm flag to the command line. If there's only one copy of bowtie2 using that index on the machine, there shouldn't be any change. If there are multiple copies of bowtie2 running at the same time and using the same index, the total RAM use will be less as the operating system will only have one copy of the index in memory. You shouldn't notice any difference otherwise.

Yes, this only helps if you're running multiple instances of bowtie2 at once.