Masurca 3.3.0 3-overlap correction errors

[ksawatzki]

I've been struggling through running Masurca with PE Illumina + ONT data for a mammalian genome on a SLURM cluster for months now. I'm completely stumped by what seemed to be a straightforward segmentation fault during overlap trimming (3-*).

Summary is:

1. Fragment correction completes without error (on 31 jobs)
2. Overlap correction completes 62 jobs with 2 types of non-fatal errors: 1. "Bad alignment" but end with "Finished" message, which I don't think I need to worry about and 2. 45 jobs fail with segmentation fault which results in no job erate (completed) file which I am very worried about.
3. I have re-run 3x by removing/restarting from ovlcor.sh, 3-* and 1-* and had the same result.
4. I have run ovlcor.sh jobs by hand. They have started and proceeded as before, then seg faulted. The reason I do not think this is a memory issue is because I ran a single job using 1TB memory (lol) and it still had this error.
5. These errors are inexplicably non-fatal and assembly will continue, despite these jobs never being re-run or fixed.

Has anyone else run into this type of error and how did you fix it? It's driving me crazy!

I've included the error file (with redacted dirs) for one of the jobs below:

Quality Threshold = 1.50%
Allocating 42987688 words for Edit_Space.
Starting Read_Frags ()
Starting Correct_Frags ()
Starting Read_Olaps ()
Starting qsort ()
Starting Redo_Olaps ()
ERROR: Bad alignment ends a_end = 0 b_end = 0
a_iid = 67359 b_iid = 1876503 errors = 0
ERROR: Bad alignment ends a_end = 0 b_end = 0
a_iid = 3385 b_iid = 1904069 errors = 0

Failed with 'Segmentation fault'

Backtrace (mangled):

/cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps[0x405f85]
/lib64/libpthread.so.0[0x38a8a0f7e0]
/lib64/libc.so.6(fwrite+0x34)[0x38a82677e4]
/cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps[0x4076c3]
/cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps[0x406d71]
/cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps[0x403430]
/cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps[0x404c2f]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x38a821ed1d]
/cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps[0x402339]

Backtrace (demangled):

[0] /cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps() [0x405f85]
[1] /lib64/libpthread.so.0() [0x38a8a0f7e0]
[2] /lib64/libc.so.6:null) + 0x34 [0x38a82677e4]
[3] /cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps() [0x4076c3]
[4] /cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps() [0x406d71]
[5] /cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps() [0x403430]
[6] /cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps() [0x404c2f]
[7] /lib64/libc.so.6:null) + 0xfd [0x38a821ed1d]
[8] /cluster/.../MaSuRCA-3.3.0/CA8/Linux-amd64/bin/correct-olaps() [0x402339]

GDB:


/cluster/.../CA.mr.41.15.15.0.02/3-overlapcorrection/ovlcorr.sh: line 45: 13264 Segmentation fault (core dumped) $bin/correct-olaps -S /cluster/.../CA.mr.41.15.15.0.02/genome.ovlStore -e /cluster/.../CA.mr.41.15.15.0.02/3-overlapcorrection/$jobid.erate.WORKING /cluster/.../CA.mr.41.15.15.0.02/genome.gkpStore /cluster/.../CA.mr.41.15.15.0.02/3-overlapcorrection/genome.frgcorr $frgBeg $frgEnd

Edit: It's hilarious that my error message includes an encoded sad face and I'm keeping it.