《A Distributed Graph Engine for Web Scale RDF Data》2013——筆記

ABSTRACT

現有系統無法有效處理Web規模的RDF數據，不支持對RDF數據的許多有用和通用的基於圖形的操作。本文使用Trinity.RDF，以原始圖形式存儲RDF數據，而不是三元組或者位圖矩陣。

Introduction

RDF數據越來越多。
數據庫管理系統面臨兩個挑戰：systems’ scalability and generality.
1.目前以三元組爲形式並使用RDBMS進行存儲，索引和查詢處理，可拓展性不強，因爲處理查詢經常涉及大量的中間結果的連接操作。
2.現有系統不支持RDF數據的通用查詢。大部分僅針對SPARQL進行優化。但是SPARQL查詢有缺陷，比如可達性查詢，random walks。
Overview of Our Approach
Trinity.RDF——a distributed in-memory RDF system
構建在內存雲之上，並以圖的形式保存。
它不僅爲SPARQL查詢處理帶來了新的優化機會，而且還支持RDF數據更高級的圖形分析。
目前大部分對圖的操作完全依賴隨機訪問，所以以三元組的形式存儲到磁盤上不妥，因爲硬盤的隨機訪問很慢，通過索引增加訪問速度的同時，也引入了過多的連接操作。
優點1：將RDF數據保存爲內存中的圖，提高了隨機訪問的效率。
優點2：利用內存內的圖搜索，減少連接操作，和中間結果的數量。
Specifically, we decompose a SPARQL query into a set of triple patterns, and conduct a sequence of graph explorations to generate bindings for each of the triple pattern. The exploration-based approach uses the binding information of the explored subgraphs to prune candidate matches in a greedy manner.
優點3：可以對RDF數據進行大量的高級圖表分析，如random walks, regular expression queries, reachability queries, distance oracles, community searches。
本文貢獻：
1. We introduce a novel graph-based scheme for managing RDF data. Trinity.RDF has the potential to support efficient graph-based queries, as well as advanced graph analytics, on RDF.
2. We leverage graph exploration for SPARQL processing. The new query paradigm greatly reduces the volume of intermediate results, which in turn boosts query performance and system scalability.
3. We introduce a new cost model, novel cardinality estimation techniques, and optimization algorithms for distributed query plan generation. These approaches ensure excellent performance on web scale RDF data.

2 Join vs. Graph Exploration

2.1 RDF and SPARQL

2.2 Using Join Operations

兩個階段：掃描階段，連接階段。並舉例圖1，表1。
雖然有優化操作，但是問題無法避免：1、大量的連接操作；2、大量中間結果。
Sideways Information Passing（SIP）生成過濾器處理類似的結果。

2.3 Using Graph Explorations

舉例說明圖搜索的優點，根據圖的某個邊一步步的擴展。可以剪枝不必要的結果。
但是以relational tables,triple stores, disk-based key-value stores存儲的不比連接有效。
另外搜索順序很重要。
闡述graph exploration 和 index-nested-loops join的區別。

3 System Architecture

Trinity.RDF is based on Trinity [30], which is a distributed in-memory key-value store. Trinity.RDF builds a graph interface on top of the key-value store. It randomly partitions an RDF graph across a cluster of commodity machines by hashing on the nodes.
然後並行的在每個服務器上進行查詢，服務器交換數據以得到完整的結果。

講述查詢的流程，用戶提交查詢，然後代理服務器，生成查詢計劃，交給其他服務器執行，然後彙總中間結果，返回用戶。
代理服務器作用：
First, it generates a query plan based on available statistics and indices. Second, it keeps track of the status of each Trinity machine in query processing by, for example, synchronizing the execution of each query step.
字符索引服務器：字符串與id的映射。

4 Data Modeling

4.1 Modeling Graphs

把RDF實體表示爲鍵值對：

(node-id, < in-adjacency-list, out-adjacency-list>) (1)

給定任意節點，我們可以找到其任意鄰居的node_id，並且底層的Trinity內存雲將該節點的鍵值對。這是我們能夠通過訪問途中任意頂點的鄰接列表來搜索圖。

4.2 Graph Partitioning

在搜索圖時，兩個因素會影響網絡開銷。
一是如何分割圖。對節點id進行hash，即隨機劃分。
二是如何在鍵值存儲的頂部建立圖形。因爲一些頂級頂點會有很多鄰居，所以傳輸該節點集會造成巨大的開銷。
通過以下鍵值對對節點x進行建模：

(node-id,<in1,...,ink,out1,...,outk >) (2)
where ini and outi are keys to some other key-value pairs:
(ini , in−adjacency−listi ) (outi , out−adjacency−listi ) (3)

4.3 Indexing Predicates

Graph exploration relies on retrieving nodes connected by an edge of a given predicate.
Local predicate indexing : 對應傳統RDF中的SPO或OPS索引，另外添加了聚合索引。
Global predicate indexing ： The global predicate index enables us to find all nodes that have incoming or outgoing neighbors labeled by a given predicate.對應於PSO或POS索引。另外對於每個謂詞還有一個鍵值對。
(predicate,

4.4 Basic Graph Operators

三種圖形運算符(dir: direction，即入向還是出向)：
1.LoadNodes (predicate, dir): Return nodes that have an incoming or outgoing edge labeled as predicate.利用全局謂詞變量在所有機器上找尋節點。
2.LoadNeighborsOnMachine(node, dir, i): For a given node, return its incoming or outgoing neighbors that reside on machine i. 找某個機器某個節點的所有incoming or outgoing neighbors，返回id。
3.SelectByPredicate(nid,predicate): From a given partial adjacency list specified by nid, return nodes that are labeled with the given predicate.

5 Query Processing

5.1 Overview

把SPARQL查詢化爲子圖匹配問題。
利用內存中的基於鍵值存儲的快速查詢處理。
步驟：
1、拆分Q到一些列的三元組q1,...,qn .
2、對每個qi 找到匹配，並拓展到qi+1 。以此類推
3、在代理服務器彙總中間結果。

5.2 Single Triple Pattern Matching

匹配從一個三元模式開始。
對triple pattern q我們找到所有匹配的R(q)。令P表示q中的謂詞，V表示q中的變量，B(V)表示V的binding（可能取值）。
兩種圖搜索的方式：
q→ 從主到賓
q← 從賓到主

算法一用4.4節定義的運算符進行圖匹配。
首先根據謂詞確定所有的源點（src）。
然後針對每個源點s，遍歷所有機器得到所有的（s,nidi ）組合，併發送到所有的機器。
然後遍歷所有（s,nidi ）組合，對於所有的謂詞，找到目標的可能值，加入到結果中。

5.3 Multiple Pattern Matching by Exploration

Instead of matching single patterns independently, we treat the query as a sequence of patterns. The matching of the current pattern is based on the matches of previous patterns, i.e., we “explore” the RDF graph from the matches of previous patterns to find matches for the current pattern. In other words, we eagerly prune invalid matchings by exploration to avoid the cost of joining large sets of results later.
按順序對三元組進行查詢，在每一步中可以修建無效的匹配。
在第一個情況，the source of exploration is bound. 除了第一步，源的取值由上一步的查詢結果來確定。
在第二個情況，the target of exploration is bound.

5.4 Final Join after Exploration

剪枝了源點，則目標點也同樣被剪枝，大大減少了中間結果大小。
但是剪枝後的結果也會有無效的中間結果，需要攜帶之前節點的所有可能值，增加了溝通成本。
our join phase is light-weight compared with traditional RDF systems that intensively rely on joins, and we simply adopt the left-deep join for this purpose.

5.5 Exploration Plan Optimization

把圖搜索定義爲圖遍歷計劃，即<e1,...,en> , ei=qi→ or ei=qi← ，總共的成本在於ei 的順序。
和關係查詢優化器調整join順序的類似，但不相同。
In the relational optimizer, later joins depend on previous intermediary join results, while for us, later explorations depend on previous intermediary bindings. The intermediary join results do not depend on the order of join, while the intermediary bindings do depend on the order of exploration.
這個中間結果會隨順序不同而變化。
There are two ways to grow a subgraph: expansion and combination.
ϵ 表示圖
R(ϵ) 表示中間連接結果
B(ϵ) 圖中的可能取值。
exploration point：不包含冗餘值的點。

Heuristic 1. We expand a subgraph from its exploration point. We combine two subgraphs by connecting their exploration points.

因爲啓發式規則1，有以下特性：

Property 1. We expand a subgraph or combine two subgraphs through an edge. The two nodes on both ends of the edge are valid exploration points in the new graph.

利用動歸，初試狀態子圖大小爲1，即只有單個邊，
如果通過邊q=c⇝v , 那麼有兩種狀態：

(ϵ∪{q},v)and(ϵ∪{q},c)

用C代表擴展之前的cost，C′ 代表擴展之後的，則遞推式：

C′=min{C′,C+cost(q→)}

如果通過邊q=c1⇝c2 , 來連接兩個狀態(ϵ1,c1) 和(ϵ2,c2) 那麼有兩種狀態：

(ϵ1∪ϵ2∪q,c1)and(ϵ1∪ϵ2∪q,c2)

用C代表擴展之前的cost，C′ 代表擴展之後的，則遞推式：

C′=min{C′,C1+C2+cost(q→)}

Theorem 1. For a query graph G(V, E), the DP has time complexity O(n·|V |·|E|) where n is the number of connected subgraphs in G.
Theorem 2. Any acyclic query Q with query graph G is guaranteed to have an exploration plan.

Discussion: 沒有考慮兩種情況：
1、被查詢圖是循環的。
這個可以通過複製一些變量來打破循環。
2、被查詢圖包含謂詞連接。

5.6 Cost Estimation

首先提出Stocker 的方法，認爲s，p，o是相互獨立的，每個三元組的選擇是三個選擇的結果。
RDF-3X提出兩種：一、認爲三元組間獨立，花費依賴於連接操作。二、找出常用的連接路徑，並統計。
本文計算cost(e) 即cost(q→) ，即計算結果的大小即|R(q)|。
During exploration, we send bindings and ids of adjacency lists across network, so we measure communication cost as the binding size of the source node of the exploration, i.e. |B(src)|.

|R(q)|=|B(src)|CpCp(src),|R(tgt)|=|B(src)|Cp(tgt)Cp(src)

for expansion, assume we expand through a new edge p2 from variable x which is already connected with p1. Assume the original binding size of x is Nx . We have the new binding size N′x as

N′x=NxCP1P2CP1

The second case is combining two edges p1 and p2 on x. Assume the original binding sizes of x with predicate p1 and predicate p2 are Nx,1 and Nx,2 respectively. We have the new binding size N′x as

N′x=Nx,1Nx,2CP1P2CP1CP2

6 Evaluation

Systems We compare Trinity.RDF with centralized RDF-3X [27] and BitMat [8], as well as distributed MapReduce-RDF-3X (a Hadoop-based RDF-3X solution [20]).
Datasets
Join vs. Exploration We compare graph exploration (Trinity.RDF) with scan-join (RDF-3X and BitMat) on DBPSB and LUBM-160 datasets. The experiment results show that Trinity.RDF outperforms RDF-3X and BitMat; and more importantly, its superiority does not just come from its inmemory architecture, but from the fact that graph exploration itself is more efficient than join.

*Performance on Large Datasets