reduceByKey(_ ++ _)

很靈性的用法

val y = sc.textFile("file:///root/My_Spark_learning/reduced.txt")

y.map(value=>value.split(","))
  .map(value=>(value(0),value(1),value(2)))
  .collect
  .foreach(println)

(1,2,3)
(1,3,4)
(4,5,6)
(7,8,9)

y.map(value=>value.split(","))
  .map(value=>(value(0),Seq(value(1),value(2)).toSet))
  .reduceByKey(_ ++ _)
  .collect
  .foreach(println)

(1,List(2, 3, 4))
(4,List(5, 6))
(7,List(8, 9))

 

原文鏈接:https://stackoverflow.com/questions/44126296/what-is-the-meaning-for-reducebykey