Redis報錯-java.net.SocketException: Broken pipe (Write failed)

異常原因:

在SparkStreaming中使用Redis進行高併發寫,發現有時候會出現以下異常:

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Broken pipe (Write failed)
    at redis.clients.jedis.Connection.flush(Connection.java:303)
    at redis.clients.jedis.Connection.getMany(Connection.java:317)
    at redis.clients.jedis.Pipeline.sync(Pipeline.java:98)
    at redis.clients.jedis.Pipeline.clear(Pipeline.java:84)
    at redis.clients.jedis.Pipeline.close(Pipeline.java:157)
    at com.amt.spark.streaming.UserBehaviorStreaming$1$12$1.call(UserBehaviorStreaming.java:353)
    at com.amt.spark.streaming.UserBehaviorStreaming$1$12$1.call(UserBehaviorStreaming.java:331)
    at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:225)
    at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:225)
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Broken pipe (Write failed)
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
    at redis.clients.jedis.util.RedisOutputStream.flushBuffer(RedisOutputStream.java:52)
    at redis.clients.jedis.util.RedisOutputStream.flush(RedisOutputStream.java:133)
    at redis.clients.jedis.Connection.flush(Connection.java:300)

 

解決過程:

百度谷歌嘗試了各種方法,發現一點用都沒有,異常原因給我引導到錯誤的排查方式去了。

 

解決方法:

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章