[need help fastx_trimmer]*** buffer overflow detected ***: fastx_trimmer terminated

[sunfuhui]

Hi all,
I run into this error when I try to use fastx_trimmer.
Is it an error of my own system or fastx toolkit?
Thanks a lot in advance!

*** buffer overflow detected ***: fastx_trimmer terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x3f0e7094c7]
/lib64/libc.so.6[0x3f0e707680]
/lib64/libc.so.6(__fgets_chk+0x15b)[0x3f0e7079ab]
fastx_trimmer[0x4019e5]
fastx_trimmer[0x400d8a]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x3f0e621735]
fastx_trimmer[0x400fb1]
======= Memory map: ========
00400000-00404000 r-xp 00000000 08:03 21769374 /usr/bin/fastx_trimmer
00603000-00604000 r--p 00003000 08:03 21769374 /usr/bin/fastx_trimmer
00604000-00605000 rw-p 00004000 08:03 21769374 /usr/bin/fastx_trimmer
00605000-00632000 rw-p 00000000 00:00 0
01602000-01623000 rw-p 00000000 00:00 0 [heap]
3f0e200000-3f0e220000 r-xp 00000000 08:03 21758567 /usr/lib64/ld-2.15.so
3f0e41f000-3f0e420000 r--p 0001f000 08:03 21758567 /usr/lib64/ld-2.15.so
3f0e420000-3f0e421000 rw-p 00020000 08:03 21758567 /usr/lib64/ld-2.15.so
3f0e421000-3f0e422000 rw-p 00000000 00:00 0
3f0e600000-3f0e7ac000 r-xp 00000000 08:03 21758568 /usr/lib64/libc-2.15.so
3f0e7ac000-3f0e9ac000 ---p 001ac000 08:03 21758568 /usr/lib64/libc-2.15.so
3f0e9ac000-3f0e9b0000 r--p 001ac000 08:03 21758568 /usr/lib64/libc-2.15.so
3f0e9b0000-3f0e9b2000 rw-p 001b0000 08:03 21758568 /usr/lib64/libc-2.15.so
3f0e9b2000-3f0e9b7000 rw-p 00000000 00:00 0
3f0fe00000-3f0fe15000 r-xp 00000000 08:03 21758841 /usr/lib64/libgcc_s-4.7.0-20120507.so.1
3f0fe15000-3f10014000 ---p 00015000 08:03 21758841 /usr/lib64/libgcc_s-4.7.0-20120507.so.1
3f10014000-3f10015000 rw-p 00014000 08:03 21758841 /usr/lib64/libgcc_s-4.7.0-20120507.so.1
7f928e676000-7f928e679000 rw-p 00000000 00:00 0
7f928e69e000-7f928e6a2000 rw-p 00000000 00:00 0
7fffd962e000-7fffd964f000 rw-p 00000000 00:00 0 [stack]
7fffd977a000-7fffd977b000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

[jimmybee]

That looks like you've overloaded your memory

[sunfuhui]

thank you for your reply.=)
So it is not the problem of fastx.
What should I do to my computer?

[westerman]

If it is a memory problem (and I am not 100% convinced of that -- buffer overflows while occurring in memory are not memory size problems; see the wikipedia article) then one thing you could do is to break up your input file into smaller files and then process these individually.

[Luka]

Hi,

I am experiencing the same problem with every Fastax tool, did anyone understand what is a possible reason for this?
I have to mention that I am performing this remotely, could that introduce some restrictions to the tools?

Any suggestion and comment is highly appreciated.

[westerman]

I have to mention that I am performing this remotely, could that introduce some restrictions to the tools?

Remote execution should not have an effect.

To better troubleshoot your problem we will need to know more.
1) You say that this occurs with every FastX tool. Does it occur when do something simple like display the help screen? Or does it only occur when you are actually processing data?

2) If the latter then try a really simple tool. I suggest fastx_renamer.

3) If the problem continues then what are the sizes of your input files? Size of your computer's memory?

4) If you can include the error output then that would be useful.

[Luka]

Hi, thank you for you prompt reply.

1) It occurs only when I process data. The help screen is displayed normally.
2)I did actually as I thought the same, but fastx_renamer gives out the same message.
3)To avoid that problem I actually created a test fastaq files, containing only 4 sequences.
4) I can include it easily, it is quite identical to the one in the opening post:
clipper

*** buffer overflow detected ***: fastx_clipper terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x3516f094c7]
/lib64/libc.so.6[0x3516f07680]
/lib64/libc.so.6(__fgets_chk+0x15b)[0x3516f079ab]
fastx_clipper[0x403255]
fastx_clipper[0x401c1a]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x3516e21735]
fastx_clipper[0x402061]
======= Memory map: ========
00400000-0040d000 r-xp 00000000 fd:01 567837 /usr/bin/fastx_clipper
0060c000-0060d000 r--p 0000c000 fd:01 567837 /usr/bin/fastx_clipper
0060d000-0060e000 rw-p 0000d000 fd:01 567837 /usr/bin/fastx_clipper
0060e000-0063b000 rw-p 00000000 00:00 0
0080d000-0080f000 rw-p 0000d000 fd:01 567837 /usr/bin/fastx_clipper
02118000-02139000 rw-p 00000000 00:00 0 [heap]
3516a00000-3516a20000 r-xp 00000000 fd:01 568782 /usr/lib64/ld-2.15.so
3516c1f000-3516c20000 r--p 0001f000 fd:01 568782 /usr/lib64/ld-2.15.so
3516c20000-3516c21000 rw-p 00020000 fd:01 568782 /usr/lib64/ld-2.15.so
3516c21000-3516c22000 rw-p 00000000 00:00 0
3516e00000-3516fac000 r-xp 00000000 fd:01 568783 /usr/lib64/libc-2.15.so
3516fac000-35171ac000 ---p 001ac000 fd:01 568783 /usr/lib64/libc-2.15.so
35171ac000-35171b0000 r--p 001ac000 fd:01 568783 /usr/lib64/libc-2.15.so
35171b0000-35171b2000 rw-p 001b0000 fd:01 568783 /usr/lib64/libc-2.15.so
35171b2000-35171b7000 rw-p 00000000 00:00 0
3517200000-35172fa000 r-xp 00000000 fd:01 556374 /usr/lib64/libm-2.15.so
35172fa000-35174f9000 ---p 000fa000 fd:01 556374 /usr/lib64/libm-2.15.so
35174f9000-35174fa000 r--p 000f9000 fd:01 556374 /usr/lib64/libm-2.15.so
35174fa000-35174fb000 rw-p 000fa000 fd:01 556374 /usr/lib64/libm-2.15.so
351a600000-351a615000 r-xp 00000000 fd:01 568802 /usr/lib64/libgcc_s-4.7.2-20120921.so.1
351a615000-351a814000 ---p 00015000 fd:01 568802 /usr/lib64/libgcc_s-4.7.2-20120921.so.1
351a814000-351a815000 rw-p 00014000 fd:01 568802 /usr/lib64/libgcc_s-4.7.2-20120921.so.1
351ea00000-351eae5000 r-xp 00000000 fd:01 568817 /usr/lib64/libstdc++.so.6.0.17
351eae5000-351ece4000 ---p 000e5000 fd:01 568817 /usr/lib64/libstdc++.so.6.0.17
351ece4000-351ecec000 r--p 000e4000 fd:01 568817 /usr/lib64/libstdc++.so.6.0.17
351ecec000-351ecee000 rw-p 000ec000 fd:01 568817 /usr/lib64/libstdc++.so.6.0.17
351ecee000-351ed03000 rw-p 00000000 00:00 0
7f3961e13000-7f3961e17000 rw-p 00000000 00:00 0
7f3961e2d000-7f3961e31000 rw-p 00000000 00:00 0
7fff037b2000-7fff037d3000 rw-p 00000000 00:00 0 [stack]
7fff037ff000-7fff03800000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)

renamer,

*** buffer overflow detected ***: fastx_renamer terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x3516f094c7]
/lib64/libc.so.6[0x3516f07680]
/lib64/libc.so.6(__fgets_chk+0x15b)[0x3516f079ab]
fastx_renamer[0x4017c5]
fastx_renamer[0x400d5b]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x3516e21735]
fastx_renamer[0x400e71]
======= Memory map: ========
00400000-00404000 r-xp 00000000 fd:01 532919 /usr/bin/fastx_renamer
00603000-00604000 r--p 00003000 fd:01 532919 /usr/bin/fastx_renamer
00604000-00605000 rw-p 00004000 fd:01 532919 /usr/bin/fastx_renamer
00605000-00632000 rw-p 00000000 00:00 0
02004000-02025000 rw-p 00000000 00:00 0 [heap]
3516a00000-3516a20000 r-xp 00000000 fd:01 568782 /usr/lib64/ld-2.15.so
3516c1f000-3516c20000 r--p 0001f000 fd:01 568782 /usr/lib64/ld-2.15.so
3516c20000-3516c21000 rw-p 00020000 fd:01 568782 /usr/lib64/ld-2.15.so
3516c21000-3516c22000 rw-p 00000000 00:00 0
3516e00000-3516fac000 r-xp 00000000 fd:01 568783 /usr/lib64/libc-2.15.so
3516fac000-35171ac000 ---p 001ac000 fd:01 568783 /usr/lib64/libc-2.15.so
35171ac000-35171b0000 r--p 001ac000 fd:01 568783 /usr/lib64/libc-2.15.so
35171b0000-35171b2000 rw-p 001b0000 fd:01 568783 /usr/lib64/libc-2.15.so
35171b2000-35171b7000 rw-p 00000000 00:00 0
351a600000-351a615000 r-xp 00000000 fd:01 568802 /usr/lib64/libgcc_s-4.7.2-20120921.so.1
351a615000-351a814000 ---p 00015000 fd:01 568802 /usr/lib64/libgcc_s-4.7.2-20120921.so.1
351a814000-351a815000 rw-p 00014000 fd:01 568802 /usr/lib64/libgcc_s-4.7.2-20120921.so.1
7f43b90e3000-7f43b90e6000 rw-p 00000000 00:00 0
7f43b90fc000-7f43b9100000 rw-p 00000000 00:00 0
7fff9eb98000-7fff9ebb9000 rw-p 00000000 00:00 0 [stack]
7fff9ebff000-7fff9ec00000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)

P.s. I should also mention that other programs such as fastqc or cutadapt seem to run normally when I use them on the same files, also remotely installed on the same computer.

[westerman]

OK. This means there is something fundamentally wrong with your copy of the fastX tools and how it works on your operating system. Either you are running the wrong version for your operating system or the compilation was done incorrectly. I am not sure if you installed the tools or if you had a sysadmin do the work but in either case I suggest wiping out your current versions of fastx and then very carefully following the install instructions at:

http://hannonlab.cshl.edu/fastx_toolkit/download.html

If you just picking up a pre-compiled version and having problems then you will instead have to go the harder route and pick up the source version and compile that.

[Luka]

Thank you for your suggestion, but I am not sure it can be useful in this case. These tools are installed on a common system an I have been accessing them remotely on my VM. Thus, I believe the installation was done properly and they do actually work, and however I do not have possibility to do what suggested.

I was simply hoping that this problem that is occurring only locally might be easy to solve, if that is not the case, I do fully understand.

[westerman]

Thank you for your suggestion, but I am not sure it can be useful in this case. These tools are installed on a common system an I have been accessing them remotely on my VM. Thus, I believe the installation was done properly and they do actually work....

So then the question becomes "does anyone else who uses the common system have the same problems that you do?" If no one else is having the problem then the fault is with your files (please send your test 4-sequence file). If other people are having the same problem then the installation was done improperly.

I was simply hoping that this problem that is occurring only locally might be easy to solve, ...

It does make it easy. The problem is with the install of FastX or with your files. I am very certain that the problem is not with FastX per se. In the "install problem" case, since you have not install privileges on the machine then you just need to pester the person who does. In the latter case ("file problem") then we'll just need to fix your files.

[GenoMax]

Thank you for your suggestion, but I am not sure it can be useful in this case. These tools are installed on a common system an I have been accessing them remotely on my VM. Thus, I believe the installation was done properly and they do actually work, and however I do not have possibility to do what suggested.

Just for clarification you are accessing (cross mounting?) a file system from a central server (something like "andrew file system") and using the tools installed in that space but actually executing them on your local VM? What OS are you using on your VM?

[westerman]

Ah, good point GenoMax. I had not considered that possibility that he would mount the tools from a different system than the one he is running on. In any case that is an "installation problem" as well. A person can not expect to use a, say, Windows program on a Mac. Or even a Linux program on an different version of Linux.

Luka. We need to know more details about what system you are using.

[Luka]

Let me try to answer your questions accurately, as you might have noticed my knowledge of this is limited.
The VM I access remotely is using Unix with the Gnome platform, as far as I understand this is identical to the file system where my remote home is located.

And to answer also the previous question, this is the sequence format in my test file

@M01132:5:000000000-A1WDR:1:1112:15205:6427 1:N:0:1
TGAGTGACCACTTACCTGAGGAGACGGTGACCAGGGTTCCCTGGCCCCAGTAGGGGTAACCACTCCAAAAATCGTATAGGAGGTAAGAGCATCTCGCACAGTAATACACGGCCGTGTCGTCAGATCTCAGGCTCCTCAGCTCCATGTAGGC
+
??A??BBBDDDDDDDDGFFGGFHIIHHHIIHHHFFFFFHHHHHHGHHHHHEGHHID5CFHHIIIIHHIIHHHHHFHHHHHHIHHHHHHHHHHHHHHHHHEFHFFHHHHGEBEGGEGGGEGGGGEGGGGGEECAEGGGECGEGGGGG:CE:?

[GenoMax]

The VM I access remotely is using Unix with the Gnome platform, as far as I understand this is identical to the file system where my remote home is located.

I think this may point to the problem. The VM may not have enough RAM allocated to it.

How big are the sequence files that you are trying to work with BTW?

Can you run the command "free -g" (without quotes of course) while logged into your VM and send the output? That should tell us how much RAM is allocated to your VM.

[westerman]

1) Luke: The 1-sequence file works fine for me.

2) GenoMax. Simple fastX programs (e.g., renamer, trimmer) should not take up much memory. Also he is having problems with a 4-sequence file. I really think that the problem is that he is using a version of fastX tools that is incompatible with his operating system. Note that the problem is with /lib64/libc.so.6 which is a OS-level routine.

3) Luka: Even without a sysadmin you should be able to install your own copy of fastX tools (from the web page I posted earlier). It may be frustrating but, heck, welcome to the "wonderful" world of bioinformatics.