Getting error Streaming Command failed! with mouse chromosome17 example

[poojas]

Hi ,

I have installed crossbow 1.2.1 and hadoop CDH3. The installation of crossbow is successful.

When i execute $CROSSBOW_HOME/cb_hadoop \
--preprocess \
--input=hdfs:///crossbow/example/mouse17/full.manifest \
--output=hdfs:///crossbow/example/mouse17/output_full \
--reference=hdfs:///crossbow-refs/mm9_chr17.jar

It fails after executing several times map and reduce Error streaming.StreamJob: Job not sucessful.
exiit level 256.

Streaming command failed!!

It is failing in the 1st process itself.

Please help. I would appreciate your help.

Thanks,
Pooja

[poojas]

14/03/06 21:09:27 INFO streaming.StreamJob: Tracking URL: http://r06n03-data.default.domain:50...403061550_0005
14/03/06 21:09:28 INFO streaming.StreamJob: map 0% reduce 0%
14/03/06 21:09:33 INFO streaming.StreamJob: map 9% reduce 0%
14/03/06 21:09:37 INFO streaming.StreamJob: map 91% reduce 0%
14/03/06 21:09:38 INFO streaming.StreamJob: map 100% reduce 0%
14/03/06 21:12:05 INFO streaming.StreamJob: map 73% reduce 0%
14/03/06 21:12:06 INFO streaming.StreamJob: map 55% reduce 0%
14/03/06 21:12:13 INFO streaming.StreamJob: map 91% reduce 0%
14/03/06 21:14:42 INFO streaming.StreamJob: map 73% reduce 0%
14/03/06 21:14:45 INFO streaming.StreamJob: map 64% reduce 0%
14/03/06 21:14:50 INFO streaming.StreamJob: map 91% reduce 0%
14/03/06 21:14:52 INFO streaming.StreamJob: map 100% reduce 0%
14/03/06 21:17:19 INFO streaming.StreamJob: map 91% reduce 0%
14/03/06 21:17:20 INFO streaming.StreamJob: map 82% reduce 0%
14/03/06 21:17:22 INFO streaming.StreamJob: map 100% reduce 100%
14/03/06 21:17:22 INFO streaming.StreamJob: To kill this job, run:
14/03/06 21:17:22 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=sgi-jt-data:54311 -kill job_201403061550_0005
14/03/06 21:17:22 INFO streaming.StreamJob: Tracking URL: http://r06n03-data.default.domain:50...403061550_0005
14/03/06 21:17:22 ERROR streaming.StreamJob: Job not successful. Error: NA
14/03/06 21:17:22 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
Non-zero exitlevel from Preprocess stage
elastic-mapreduce script completed with exitlevel 256



Thanks ,
PS

[poojas]

CrossBow1.2.1 And e_coli.jar issue

Hello,

After instaliing the crossbow1.2.1 and CDH3 the crosbbow installation is successful. But e_coli.jar is failing with the following messages.

Please help..


bash-4.1$ $CROSSBOW_HOME/cb_hadoop \
> --preprocess \
> --input=hdfs:///crossbow/example/e_coli/small.manifest \
> --output=hdfs:///crossbow/example/e_coli/output_small \
> --reference=hdfs:///crossbow-refs/e_coli.jar \
> --all-haploids
Cannot find streaming jar in /usr/contrib/streaming; please specify --streaming-jar
bash-4.1$ $CROSSBOW_HOME/cb_hadoop --preprocess --input=hdfs:///crossbow/example/e_coli/small.manifest --output=hdfs:///crossbow/example/e_coli/output_small --reference=hdfs:///crossbow-refs/e_coli.jar --all-haploids --streaming-jar=/usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.jar
Crossbow expects 'bowtie' to be at path /usr/local/bin/bowtie-0.12.8/bowtie on the workers
Crossbow expects 'soapsnp' to be at path /usr/local/bin/crossbow-1.2.1/soapsnp/soapsnp on the workers
Crossbow expects 'fastq-dump' to be at path /usr/local/bin/sratoolkit.2.3.4-2-centos_linux64/bin/fastq-dump on the workers

Crossbow job
------------
Hadoop streaming commands in: /tmp/crossbow-cFSkFTM86H/invoke.scripts/cb.40124.hadoop.sh
Running...
==========================
Stage 1 of 4. Preprocess
==========================
Fri Mar 7 09:02:46 PST 2014
packageJobJar: [/usr/local/bin/crossbow-1.2.1/Get.pm, /usr/local/bin/crossbow-1.2.1/Counters.pm, /usr/local/bin/crossbow-1.2.1/Util.pm, /usr/local/bin/crossbow-1.2.1/Tools.pm, /usr/local/bin/crossbow-1.2.1/AWS.pm, /tmp/hadoop-hdfs/hadoop-unjar3156726987397311824/] [] /tmp/streamjob8810576896590027837.jar tmpDir=null
14/03/07 09:02:48 INFO mapred.FileInputFormat: Total input paths to process : 1
14/03/07 09:02:48 INFO streaming.StreamJob: getLocalDirs(): [/hdfs/01/mapred/local, /hdfs/02/mapred/local, /hdfs/03/mapred/local, /hdfs/04/mapred/local, /hdfs/05/mapred/local, /hdfs/06/mapred/local, /hdfs/07/mapred/local, /hdfs/08/mapred/local, /hdfs/09/mapred/local, /hdfs/10/mapred/local]
14/03/07 09:02:48 INFO streaming.StreamJob: Running job: job_201403061550_0006
14/03/07 09:02:48 INFO streaming.StreamJob: To kill this job, run:
14/03/07 09:02:48 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=sgi-jt-data:54311 -kill job_201403061550_0006
14/03/07 09:02:48 INFO streaming.StreamJob: Tracking URL: http://r06n03-data.default.domain:50...403061550_0006
14/03/07 09:02:49 INFO streaming.StreamJob: map 0% reduce 0%
14/03/07 09:02:54 INFO streaming.StreamJob: map 67% reduce 0%
14/03/07 09:03:16 INFO streaming.StreamJob: map 100% reduce 100%
14/03/07 09:03:16 INFO streaming.StreamJob: To kill this job, run:
14/03/07 09:03:16 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=sgi-jt-data:54311 -kill job_201403061550_0006
14/03/07 09:03:16 INFO streaming.StreamJob: Tracking URL: http://r06n03-data.default.domain:50...403061550_0006
14/03/07 09:03:16 ERROR streaming.StreamJob: Job not successful. Error: NA
14/03/07 09:03:16 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
Non-zero exitlevel from Preprocess stage
elastic-mapreduce script completed with exitlevel 256
bash-4.1$

Thanks in advance.
-Pooja