这期内容当中小编将会给大家带来有关CDH火花2的迷你图2-提交的一个没有这样的文件或目录问题怎么解决，文章内容丰富且以专业的角度为大家分析和叙述，阅读完这篇文章希望大家可以有所收获。
运行：
在测试的CDH火花2,运行火花流，
命令如下：
迷你图2-提交\
-上课。com。远程导航。数据平台。演示。真实案例。天气警报\
-主纱线-展开模式集群\
/usr/local/sparkProject/realtimeCases-0。0 .1-快照。冲突
错误：
17/03/02 21:01:56 INFO集群雅伦集群调度程序：添加包含一个任务的任务集0.0
17/03/02 21:01:56 WARN net .ScriptBasedMapping:异常运行/etc/spark2/conf .cloudera。spark 2 _ on _ yaw/yaw-conf/topology。py 172。16 .102 .64
java.io.IOException:无法运行程序/etc/spark2/conf .cloudera。spark 2 _ on _ yaw/yaw-conf/topology。py '(在目录中/纱/nm/用户缓存/spark/app缓存/application _ 148845908260 _ 0003/container _ 1488459089260 _ 0003 _ 01 _ 0000001’):错误=2,没有这样的文件或目录
在Java。朗。processBuilder。启动(processBuilder。Java :1048)
在组织。阿帕奇。Hadoop。乌提尔。贝壳。运行命令(Shell。Java :548)
在组织。阿帕奇。Hadoop。乌提尔。贝壳。运行(Shell。Java :504)
在。组织。阿帕奇。Hadoop。乌提尔。shell $ shellcommandxecutor。执行(Shell。Java :786)
在组织。阿帕奇。Hadoop。净。scriptbasedmapping $ rawscriptbasedmapping。runresolvecommand(scriptbasedmapping。Java 3360251)
在。组织。阿帕奇。Hadoop。净。scriptbasedmapping $ rawscriptbasedmapping。resolve(scriptbasedmapping。Java :188)
在组织。阿帕奇。Hadoop。净。cacheddnstoswitchmapping。resolve(cacheddnstoswitchmapping。Java :119)
在组织。阿帕奇。Hadoop。纱线。乌提尔。rackresolver。CoreResolution(rackresolver。Java :101)
在。组织。阿帕奇。Hadoop。纱线。乌提尔。rackresolver。解析(rackresolver。Java :81)
at org.apache.spa
rk.scheduler.cluster.YarnScheduler.getRackForHost(YarnScheduler.scala:37)
at org.apache.spark.scheduler.TaskSetManager$$anonfun$org$apache$spark$scheduler$TaskSetManager$$addPendingTask$1.apply(TaskSetManager.scala:201)
at org.apache.spark.scheduler.TaskSetManager$$anonfun$org$apache$spark$scheduler$TaskSetManager$$addPendingTask$1.apply(TaskSetManager.scala:182)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.TaskSetManager.org$apache$spark$scheduler$TaskSetManager$$addPendingTask(TaskSetManager.scala:182)
at org.apache.spark.scheduler.TaskSetManager$$anonfun$1.apply$mcVI$sp(TaskSetManager.scala:161)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
at org.apache.spark.scheduler.TaskSetManager.<init>(TaskSetManager.scala:160)
at org.apache.spark.scheduler.TaskSchedulerImpl.createTaskSetManager(TaskSchedulerImpl.scala:222)
at org.apache.spark.scheduler.TaskSchedulerImpl.submitTasks(TaskSchedulerImpl.scala:186)
at org.apache.spark.scheduler.DAGScheduler.submitMissingTasks(DAGScheduler.scala:1058)
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:933)
at org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:873)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1632)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1624)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1613)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
解决思路:
1.分析这句话,
17/03/02 21:01:56 WARN net.ScriptBasedMapping: Exception running /etc/spark2/conf.cloudera.spark2_on_yarn/yarn-conf/topology.py 172.16.102.64
java.io.IOException: Cannot run program "/etc/spark2/conf.cloudera.spark2_on_yarn/yarn-conf/topology.py" (in directory "/yarn/nm/usercache/spark/appcache/application_1488459089260_0003/container_1488459089260_0003_01_000001"): error=2, No such file or directory
说明在这个ip的机器上 没有这个py文件。
然后去机器验证，
然后再将01机器的 配置文件 全部copy到另外四台即可。
scp -r /etc/spark2/conf.cloudera.spark2_on_yarn root@hadoop-02:/etc/spark2/
scp -r /etc/spark2/conf.cloudera.spark2_on_yarn root@hadoop-03:/etc/spark2/
scp -r /etc/spark2/conf.cloudera.spark2_on_yarn root@hadoop-04:/etc/spark2/
scp -r /etc/spark2/conf.cloudera.spark2_on_yarn root@hadoop-05:/etc/spark2/
验证:
就ok了
上述就是小编为大家分享的CDH Spark2的spark2-submit的一个No such file or directory问题怎么解决了，如果刚好有类似的疑惑，不妨参照上述分析进行理解。如果想知道更多相关知识，欢迎关注行业资讯频道。
内容来源网络，如有侵权，联系删除，本文地址：https://www.230890.com/zhan/149957.html