Spark2 Dataset之collect_set與collect_list

concat_猥瑣

concat_ws(',', collect_list(pulprb))
concat_ws(',', collect_list(pulprb))
collect_set去除重複元素；collect_list不去除重複元素



spark.sql(" select t.cgi ,concat_ws(',', collect_list(pulprb))  nloc_ncgis_prb from  ( select a.cgi ,pulprb from  nloc_ncgis_prb a left join  ckpi b on  a.n_cgi=b.cgi ) t group by t.cgi ").createOrReplaceTempView("nloc_ncgis_prb_out")