spark集羣：無法指定被請求的地址,Service 'Driver' could not bind on a random free port.

19/07/09 17:00:48 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
19/07/09 17:00:49 WARN Utils: Service 'Driver' could not bind on a random free port. You may check whether configuring an appropriate binding address.

Exception in thread "main" java.net.BindException: 無法指定被請求的地址: Service 'Driver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'Driver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
	at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
	at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:989)
	at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254)
	at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:364)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)

 

需要制定 spark.driver.bindAddress 爲 127.0.0.1 就可以解決

 

  SparkSession.Builder builder = SparkSession.builder()/*.master("local[*]")*/.appName("SparkCalculateRecommend").
                config("spark.mongodb.input.uri", "mongodb://xx:[email protected]:27018/sns.igomoMemberInfo_Spark_input")
                .config("spark.mongodb.output.uri", "mongodb://xx:[email protected]:27018/sns.igomoMemberInfo_Spark_output")
                .config("spark.driver.bindAddress","127.0.0.1")
                .config("spark.executor.memory", "1g")
                .config("es.nodes", esIpAddr)
                .config("es.port", "9200")
                .config("es.nodes.wan.only", "true");