Spark學習筆記——讀寫ScyllaDB

Scylla兼容cassandra API，所以可以使用spark讀寫cassandra的方法來進行讀寫

1.查看scyllaDB對應的cassandra版本

cqlsh:my_db> SHOW VERSION
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4]

2.查看spark和cassandra對應的版本

參考：https://github.com/datastax/spark-cassandra-connector

3.寫scyllaDB

dataset API寫scyllaDB

ds2.write
      .mode("append")
      .format("org.apache.spark.sql.cassandra")
      .options(Map("table" -> "my_tb", "keyspace" -> "my_db", "output.consistency.level" -> "ALL", "ttl" -> "8640000"))
      .save()

RDD API寫scyllaDB

ds.rdd.saveToCassandra("my_db", "my_tb", writeConf = WriteConf(ttl = TTLOption.constant(8640000), consistencyLevel = ConsistencyLevel.ALL))

4.讀scyllaDB

 

參考：通過 Spark 創建/插入數據到 Azure Cosmos DB Cassandra API