今天在centos 6.3上搭建了hadoop+hive＋spark 系統，在運行
/usr/local/spark-1.0.0/bin/spark-shell
出現找不到hive 相關的類，於是修改了spark-env.sh
在spark_classpath 裏添加了 hive 的庫路徑
export SPARK_CLASSPATH=/usr/local/spark-1.0.0/lib_managed/jars/spark-assembly-1.0.0-hadoop2.4.0.jar:/usr/local/spark-1.0.0/assembly/target/scala-2.10/mysql-connector-java-5.1.25-bin.jar:/usr/local/hive-0.13/lib/*
再次運行
/usr/local/spark-1.0.0/bin/spark-shell 又出現
java.lang.IllegalAccessError: tried to access field org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.conf from class org.apache.hadoop.hive.ql.security.ProxyUserAuthenticator
at org.apache.hadoop.hive.ql.security.ProxyUserAuthenticator.setConf(ProxyUserAuthenticator.java:40)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:365)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:278)
at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:166)
進到 /usr/local/spark-1.0.0/assembly/target/scala-2.10/org/apache/hadoop/hive/ql/security 目錄下，把authorization 目錄修改成另外一個名字 ，再次運行就成了。 原因應該是類重複了（猜的，呵呵）
還有一個很重要的問題，spark 下一定要有對應hive 的配置文件。