這次讀論文給了我慘痛的教訓，不做筆記是不行的，越長的論文越應該做筆記！不可怠惰！

Abstract

propose techniques for processing SPARQL queries over a large RDF graph in a distributed environment.
“partial evaluation and assembly” framework.
partial evaluation——在每個子圖中找到部分匹配的答案。
assembly——centralized and distributed.

1 Introduction

語義網數據模型，(Resource Description Framework, RDF).
數據大量增長，與之相應地，計算和存儲需求增加，超出單個機器的能力。
關於distributed evaluation of SPARQL queries over large RDF datasets 大體上有三類方法。
1) Cloud-based approaches maintain a large RDF graph using existing cloud computing platforms, such as Hadoop or Cassandra, and employ triple pattern-based join processing most commonly using MapReduce.
2) Partition-based approaches divide the RDF graph Ginto a set of subgraphs (fragments) and decompose the SPARQL query Q into subqueries. These subqueries are then executed over the partitioned data using techniques similar to relational distributed databases.
3) Federated approaches. Federated SPARQL processing systems evaluate queries over multiple SPARQL endpoints. These systems typically target LOD and follow a query processing over data integration approach.

本文的策略是劃分圖但不拆解查詢。RDF圖被拆解成頂點不相交的段。每個站點都接受完整的查詢，並行計算。該方法是首次應用於該方向，中間結果少。

基於Partial Evaluation的分佈式數據管理中，每個機器將存儲在自身的數據視爲已知的部分s，而存儲在其他機器上的數據視爲未知的部分d。然後，每個機器利用已知部分對查詢求出部分解。最後，這些局部匹配被收集起來並通過連接操作拼成最終解。—— 來自彭鵬的知乎回答
Within the context of graph processing, the technique has been used to evaluate reachability queries and graph simulation over graphs.
但是 SPARQL is based on graph homomorphism. SPARQL query semantics is different than these (上文中的可達性查詢和圖同構).

Some SPARQL query matches are contained within a fragment, which we call inner matches.
Subgraph matches that cross multiple fragments are called crossing matches.
該框架主要需要解決兩個問題：一是計算在每個站點中查詢Q的部分評價結果。二是組合這些部分匹配的結果得到答案。
方法的優點有兩方面：
一是不依賴於特定的圖的劃分策略。
二是保證中間結果的點和邊比別的方法少。

2.1 Distributed SPARQL query processing

2.1.1 Cloud-based approaches

HDFS-based approaches

他們將RDF三元組以flat files的格式存儲到HDFS中。進行SPARQL查詢的時候，先掃一遍HDFS文件，然後使用MapReduce進行連接。這些方法的不同主要在於如何將RDF三元組存儲到HDFS文件中。
SHARD 直接存到一個文件中，每行表示一個特定主體的所有三元組。
HadoopRDF and PredicateJoin 基於謂詞劃分三元組，每個劃分存到一個文件中。
EAGRE 將具有相似屬性的主體合成一個實體類，然後構造一個僅包含實體類及其之間連接的壓縮RDF圖。使用METIS算法劃分圖。

No HDFS-based approaches

Besides the HDFS-based approaches, there are also some works that use other NoSQL distributed data stores to manage RDF datasets.
JenaHBase and H2 RDF 利用主體、客體、謂詞的排列建立索引並存儲到HBase中。
Trinity.RDF uses the distributed memory-cloud graph system Trinity to index and store the RDF graph.

基於雲的方法得益於雲平臺的高可擴展性和容錯能力，但是因MapReduce難以適配到圖計算中導致性能較低。

2.1.2 Partition-based approaches

如上文所述，就是把圖和查詢都分解。但因分解方法的不同，查詢的處理方法也不同。
GraphPartition 通過邊界點的N條鄰居來得到每個段。
WARP uses some frequent structures in workload to further extend the results of GraphPartition.
Partout…
TriAD METIS 劃分RDF圖。並且結果的塊數比site數要多。

但是有一些情況是需要根據特定的需求來劃分數據的。這些根據方法去劃分的方法不夠靈活。

2.1.3 Federated SPARQL query systems

Federated queries run SPARQL queries over multiple SPARQL endpoints. A typical example is linked data, where different RDF repositories are interconnected, providing a virtually integrated distributed database. (與文章不相關，我不做筆記了)

2.2 Partial evaluation

與第一章類似，介紹了部分評價的應用，如在分佈式XML中進行XPath查詢，圖的同構查詢等等。詳細介紹了圖同構（多項式時間可解）與圖同態（完全NP問題）的區別。

3 Background and framework

Definition 1
(RDF graph) An RDF graph is denoted as G={V,E,Σ} , where V is a set of vertices that correspond to all subjects and objects in RDF data; E⊆V×V is a multiset of directed edges that correspond to all triples in RDF data; Σ is a set of edge labels. For each edge e∈E , its edge label is its corresponding property.

同樣的，SPARQL查詢同樣可以表示爲圖。我們首次關注basic graph pattern (BGP) qurries。在第六章有詳細的討論。
Definition 2
(SPARQL BGP query) A SPARQL BGP query is denoted as Q={VQ,EQ,ΣQ} , where VQ⊆V∪VVar is a set of vertices, where V denotes all vertices in RDF graph G and VVar is a set of variables; EQ⊆VQ×VQ is a multiset of edges in Q; each edge e in EQ either has an edge label in Σ or the edge label is a variable.
Definition 3
(SPARQL match) Consider an RDF graph G and a connected query graph Q that has n vertices {v1,..,vn} . A subgraph M with m vertices {u1,..,um} (in G) is said to be a match of Q if and only if there exists a functionf from {v1,...,vn} to {u1,..,um}(n≥m) , where the following condition hold:

if vi is not a variable, f(vi) and vi have the same URI or literal value (1≤i≤n) ;

if vi is a variable, there is no constraint over f(vi) except that f(vi)∈{u1,..,um} ;

if there exists an edge vivj−→− in Q, there also exists an edge f(vi)f(vj)−→−−−−−− in G. Let L(vivj−→−) denote a multi-set of labels between vi and vj in Q; and L(f(vi)f(vj)−→−−−−−− denote a multi-set of labels between f(vi) and f(vj) in G. There must exist an injective function from edge labels in L(vivj−→−) to edge labels in L(f(vi)f(vj)−→−−−−−− . Note that a variable edge label in L(vivj−→−) can match any edge label in L(f(vi)f(vj)−→−−−−−− .

圖被劃分爲不同的段，不同的分佈式RDF系統使用不同的劃分算法，但是我們的方法對劃分算法不敏感。

Definition 4
(Distributed RDF graph) A distributed RDF graph G={V,E,Σ} consists of a set of fragments F={F1,F2,…,Fk} where each Fi is specified by (Vi∪Vei,Ei∪Eci,Σi)(i=1,...,k) such that

{V1,…,Vk} is a partitioning of V, i.e., Vi∩Vj=∅,1≤i,j≤k,i≠j and ⋃i=1,…,kVi=V

Ei⊆Vi×Vi,i=1,...,k ;

Eci is a set of crossing edges between Fi and other fragments, i.e., （交叉邊定義）

a vertex u′∈Vei if and only if vertex u′ resides in another fragment Fj and u′ is an endpoint of a crossing edge between fragment FiandFj(Fi≠Fj) ,i.e., $V e i = (⋃ 1 \leq j \leq k \land j \neq i {u' | u u' - \to \in E c i \land u \in F i}) ⋃ (⋃ 1 \leq j \leq k \land j \neq i {u' | u' u - \to \in E c i \land u \in F i})$ ;（外部點集合）

Vertices in Vei are called extended vertices of Fi , and all vertices in Vi are called internal vertices of Fi ;(內部點和外部點)

Σi is a set of edge labels in Fi

Definition 5
(Problem statement) Let G be a distributed RDF graph that consists of a set of fragments F={F1,…,Fk} and let S={S1,…,Sk} be a set of computing nodes such that Fi is located at Si . Given a SPARQL query graph Q, our goal is to find all SPARQL matches of Q in G.

段內能匹配的問題，不再考慮，主要研究跨段匹配問題。
There are three steps in our method.
Step 1 (Initialization): A SPARQL query Q is input and sent to each site in S.
Step 2 (Partial Evaluation): Each site Si finds local partial matches of Q over fragment Fi. This step is executed in parallel at each site (Sect. 4).
Step 3 (Assembly): Finally, we assemble all local partial matches to compute complete crossing matches. The system can use the centralized (Sect. 5.2) or the distributed assembly approach (Sect. 5.3) to find crossing matches.

4 Partial evaluation

4.1 Local partial match: definition

Definition 6
(Local partial match) Given a SPARQL query graph Q with n vertices{v1,...,vn} and a connected subgraph PM with m vertices{u1,...,um}(m≤n) in a fragment Fk , PM is a local partial match in fragment Fk if and only if there exists a function f:{v1,...,vn}→{u1,...,um}∪{NULL} , where the following conditions hold:

If vi is not a variable, f(vi) and vi have the same URI or literal or f(vi)=NULL

If vi is a variable, f(vi)∈{u1,...,um} or f(vi)=NULL

If there exists an edge vivj−→− in Q(1≤i≠j≤n), then PM must meet one of the following five conditions:(1) there also exists an edge f(vi)f(vj)−→−−−−−− in PM with property p, and p is the same to the property of vivj−→− ; (2) there also exists an edge f(vi)f(vj)−→−−−−−− in PM with property p, and the property of vivj−→− is a variable; (3) there does not exist an edge f(vi)f(vj)−→−−−−−− , but f(vi) and f(vj) are both in Vek ; (4) f(vi)=NULL ; (4) f(vj)=NULL ;

PM contains at least one crossing edge, which guarantees that an empty match does not qualify.

If f(vi)∈Vk (i.e., f(vi) is an internal vertex in Fk ) and ∃vivj−→−∈Q (or vjvi−→−∈Q ), there must exist f(vj)≠NULL and ∃f(vi)f(vj)−→−−−−−−∈PM (or ∃f(vj)f(vi)−→−−−−−−∈PM ). Furthermore, if vivj−→− (or vjvi−→− ) has a property p, f(vi)f(vj)−→−−−−−− (or f(vj)f(vi)−→−−−−−− ) has the dame property p.

Any two vertices vi and vj (in query Q), where f(vi) and f(vj) are both internal vertices in PM, are weakly connected in Q.
Vector [f(v1),...,f(vn)] is a serialization of a local partial match.

The basic intuition of Condition 5 is that if vertex vivi (in query Q) is matched to an internal vertex, all of vivi’s neighbors should be matched in this local partial match as well.

Definition 7
Two vertices are weakly connected in a directed graph if and only if there exists a connected path between the two vertices when all directed edges are replaced with undirected edges. The path is called a weakly connected path between the two vertices.

The correctness of our method is stated in the following propositions.

1.The overlapping part between any crossing match M and internal vertices of fragment FiFi (i=1,…,ki=1,…,k) must be a local partial match (see Proposition 1).
2.Missing any local partial match may lead to result dismissal. Thus, the algorithm should find all local partial matches in each fragment (see Proposition 2).
3.It is impossible to find two local partial matches M and M′M′ in fragment F, where M′M′ is a subgraph of M, i.e., each local partial match is maximal (see Proposition 4).

Proposition 1
Given any crossing match M of SPARQL query Q in an RDF graph G, if M overlaps with some fragment Fi , let (M∪Fi) denote the overlapping part between M and fragment Fi . Assume that (M∪Fi) consists of several weakly connected components, denoted as (M∪Fi)={PM1,...,PMn} . Each weakly connected component PMa(1≤a≤n) in (M∪Fi) must be a local partial match in fragment Fi .(證明有定義六中的六個條件即可。)
Proposition 2
The partial evaluation and assembly algorithm does not miss any crossing matches in the answer set if and only if all local partial matches in each fragment are found in the partial evaluation stage.（反證法分別證明充分性和必要性。）保證了沒有local partial matches丟失。
Proposition 3
Given the same underlying partitioning over RDF graph G, the number of involved vertices and edges in the intermediate results (in our approach) is not larger than that in any other partition-based solution.（反證法證明本方法的每個點和每個邊其他方法都需要有，否則結果不全）保證了中間結果包含的點和邊最少。
Proposition 4
Given a query graph Q and an RDF graph G, if PMi is a local partial match under function f in fragment Fi , there exists no local partial match PM′i under function f′ in Fi ,where f⊂f′

Any PMiPMi cannot be enlarged by introducing more vertices or edges to become a larger local partial match.

4.2 Computing local partial matches

部分評價的目標就是在Fi 中找到Q的點對應的點，建立起一個映射f。f可以表示爲一系列的點對。算法如下：

修改原來用作子圖匹配的gStore，用來計算local partial matches.
狀態轉移算法

5 Assembly

找到每個fragment部分評價的結果後，就是組裝這些結果。有兩種策略：
centralized——所有部分評價的匹配對送至一個站點組裝。
distributed (or parallel)——部分匹配對在多個站點同時匹配。

5.1 Join-based assembly

Definition 8(可連接的條件)
(Joinable) Given a query graph Q and two fragments Fi and Fj(i≠j) , let PMi and PMj be the corresponding local partial matches over fragments Fi and Fj under functions fi and fj . PMi and PMj are joinable if and only if following conditions hold:

There exist no vertices u and u′ in PMi and PMj , respectively, such that f−1i(u)=f−1j(u′) (沒有重合點)

There exists at least on crossing edge uu′→ such that u is an internal vertex and u′ is an extended vertex in Fi , while u is an extended vertex and u′ is an internal vertex in Fj . Furthermore, f−1i(u)=f−1j(u) and f−1i(u′)=f−1j(u′) (至少有一個公共邊)
Definition 9
(Join result) Given a query graph Q and two fragments Fi and Fj(i≠j) , let PMi and PMj be two joinable local partial matches of Q over fragments Fi and Fj under functions fi and fj , respectively. The join of PMi and PMj is defined under a new function f (donoted as PM=PMi⋈fPMj ), which is defined as follows for any vertex v in Q:

if fi(v)≠NULL∧fj(v)=NULL,f(v)←fi(v) ;

if fi(v)=NULL∧fj(v)≠NULL,f(v)←fj(v) ;

if fi(v)≠NULL∧fj(v)≠NULL,f(v)←fi(v)(Inthiscase,fi(v)=fj(v)) ;

if fi(v)=NULL∧fj(v)=NULL,f(v)←NULL ;
(這個連接沒什麼好解釋的，具體參考圖3就好了。就是把兩個f揉到一個裏面)

5.2 Centralized assembly

集中組裝，所有的PM都被送至一個最終組裝點。提出了一個迭代連接算法（算法2）。

5.2.1 Partitioning-based join processing

爲了降低連接空間。
Theorem 1
Given two local partial matches PMi and PMj from fragments Fi and Fjwithfunctions f_iand f_j,respectively,ifthereexistsaqueryvertexvwhereboth f_i(v)and f_j(v)areinternalverticesoffragments F_iand F_j,respectively, PM_iand PM_j$ are not joinable.
(如果query中的某個點對應到兩個PM中都是內部點，那這兩個PM肯定不能相連)
Definition 10
(Local partial match partitioning). Consider a SPARQL query Q with n vertices vi,...,vn . Let Ω denote all local partial matches. P=Pv1,...,Pvn is a partitioning of Ω if and only if the following conditions hold.

Each partition P_{vi}(i=1,…,n) consists of a set of local partial matches, each of which has an internal vertex that matches vi .

Pvi∩Pvj=∅ , where 1≤i≠j≤n

Pv1∪...∪Pvn=Ω
定義了PM的劃分方式，只有在不同塊的PM纔可能join。Algorithm 3 shows how to perform partitioning-based join of local partial matches.

5.2.2 Finding the optimal partitioning

爲找到最佳的劃分方式。先定義劃分花費用來衡量劃分方案。
Definition 11
(Join cost). Given a query graph Q with n vertices v1,...,vn and a partitioning P=Pv1,...,Pvn over all local partial matches Ω , the join cost is

C o s t (Ω) = O (\prod i = 1 i = n (| P v i | + 1))

where

|Pvi| is the number of local partial matches in

Pvi and 1 is introduced to avoid the ‘0’ element in the product.
假設每個來自不同分區的LPM都是可連接的，來計算最差情況下的花費。

Definition 12
(Optimal partitioning). Given a partitioning P over all local partial matches Ω , P is the optimal partitioning if and only if there exists no another partitioning that has smaller join cost.
(定義了最優的劃分方式)
Theorem 2
Finding the optimal partitioning is NP-complete problem.

Proof
We can reduce a 0-1 integer planning problem to finding the optimal partitioning.

We formulate the 0-1 integer planning problem as follows:

min \prod i = 0 i = n ⎛ ⎝ \sum j x j i + 1 ⎞ ⎠ st. \forall j, \sum i x j i = 1

The equivalence between the 0-1 integer planning and finding the optimal partitioning are straightforward. The former is a classical NP-complete problem. Thus, the theorem holds.
Theorem 3
Given a query graph Q with n vertices

v1,..,vn and a set of all local partial matches

Ω , let

Uvi (i=1,…,n) be all local partial matches (in

Ω ) that have internal vertices matching

vi . For the optimal partitioning

Popt=Pv1,...,Pvn where

Pvn has the largest size (i.e., the number of local partial matches in

Pvn is maximum) in

Popt,Pvn=Uvn
問題在於如何利用定理3找到一個最優的序列。

C o s t (Ω) o p t = = | P v k 1 | \times C o s t (Ω v k 1 ¯ ¯ ¯ ¯) o p t | U v k 1 | \times C o s t (Ω v k 1 ¯ ¯ ¯ ¯) o p t

但是我們不知道哪一個是

vk1 引入以下結構：

C o s t (Ω) o p t = = M I N 1 \leq i \leq n (| P v i | \times C o s t (Ω v i ¯) o p t) M I N 1 \leq i \leq n (| U v i | \times C o s t (Ω v i ¯) o p t)

上式可以利用中間結果，再優化：

C o s t (Ω) o p t = = M I N 1 \leq i \leq n; 1 \leq j \leq n; i \neq j (| P v i | \times | P v j | \times C o s t (Ω v i v j ¯ ¯ ¯ ¯) o p t) M I N 1 \leq i \leq n; 1 \leq j \leq n; i \neq j (| U v i | \times | U' v j | \times C o s t (Ω v i v j ¯ ¯ ¯ ¯) o p t)

5.2.3 Join order

如果最優的劃分確定了，則連接順序也確定了。If the optimal partitioning is Popt={Pvk1,…,Pvkn} and |Pvk1|≥|Pvk2|≥…≥|Pvkn| , then the join order must be Pvk1⋈Pvk2⋈…⋈Pvkn . 原因如下：
First, changing the join order may not prune any intermediate results.
Second, in some special cases, the join order may have an effect on the performance.

5.3 Distributed assembly

We adopt Bulk Synchronous Parallel (BSP) model to design a synchronous algorithm for distributed assembly. A BSP computation proceeds in a series of global supersteps, each of which consists of three components: local computation, communication and barrier synchronization.

5.3.1 Local computation

Consider the mth superstep. For each fragment Fi , let Δmin(Fi) denote all received intermediate results in the mth superstep and Ωm(Fi) denote all local partial matches and the intermediate results generated in the first (m−1) supersteps.

5.3.2 Communication

爲了防止兩個fragment互發PM導致重複結果，需根據fragment中的PM數量進行排序。
Definition 13
Given any two fragments FiandFj , Fi≺Fj if and only if |Ω(Fi)|≤|Ω(Fj)|(1≤i,j≤n) .
採用分治法。
Assume that PM is generated by joining intermediate results from m different fragments Fi1,…,Fim , where Fi1≺Fi2≺…≺Fim . We send PM to another fragment Fj if and only if two conditions hold: (1) Fj>Fim ; and (2) Fj shares common crossing edges with at least one fragment of Fi1,...,Fim .

5.3.3 Barrier synchronization

第m步的溝通必須在第m+1步之前完成。
初始狀態的時候，只有本地的匹配結果，但不可能連接，所以不需要計算，直接進入communication階段。直接發送ΩFi 到別的fragment。

5.3.4 System termination condition

BSP算法的關鍵在於終止系統時的superstrps的數量。
Definition 14
(Fragmentation topology graph) Given a fragmentation F over an RDF graph G, the corresponding fragmentation topology graphT is defined as follows: Each node in T is a fragment Fi,i=1,...,k. There is an edge between nodes FiandFj in T, 1≤i≠j≤n , if and only if there is at least one crossing edge between FiandFj in RDF graph G.
Let Dia(T) be the diameter of T. Hence, the number of the supersteps in the BSP-based algorithm is Dia(T) .

6 Handling general SPARQL

目前只討論了basic graph pattern (BGP) query evaluation. 本節討論如何擴展至general SPARQL queries involving UNION, OPTIONAL and FILTER statements.
DeFinition 15
(General SPARQL query) Any BGP is a SPARQL query. If Q1 and Q2 are SPARQL queries, then expressions (Q1ANDQ2),(Q1UNIONQ2),(Q1OPTQ2)and(Q1FILTERF) are also SPARQL queries.
Definition 16
(Match of general SPARQL query) Given an RDF graph G, the match set of a SPARQL query Q over G, denoted as [[Q]], is defined recursively as follows:

If Q is a BGP, [[Q]] is the set of matches defined in Definition 3 of Section 3.

If Q=Q1ANDQ2 , then [[Q]]=[[Q1]]⋈[[Q2]]

If Q=Q1UNIONQ2 , then [[Q]]=[[Q1]]∪[[Q2]]

If Q=Q1OPTQ2 , then [[Q]]=([[Q1]]⋈[[Q2]])∪([[Q1]] [[Q2]])

If Q=Q1FILTERF , then [[Q]]=[[Q]]=ΘF([[Q1]])

7 Experiments

與其他方法做對比。a cloud-based approach (EAGRE), two partition-based approaches (Graphpartition and TripleGroup), two memory-based systemds (TriAD and Trinity.RDF) and two federated SPARQL query systems (FedX and SPLENDID).

7.1 Setting

We use two benchmark datasets with different sizes and one real dataset in our experiments, in addition to FedBench used in federated system experiments. Table 1 summarizes the statistics of these datasets.

1. WatDiv is a benchmark that enables diversified stress testing of RDF data management systems.把不同大小的數據集隨機分成N（N=10）份。（除了在實驗六中）
2. LUBM is a bench mark that adopts an ontology for the university domain and can generate synthetic OWL data scalable to an arbitrary size. 使用了7個benchmark查詢。
3. BTC 2012 is a real dataset that serves as the basis of submissions to Billion Triples Track of the Semantic Web Challenge.
4. FedBench is used for testing against federated systems.
實驗環境：
a cluster of 10 machines running Linux, each of which has one CPU with four cores of 3.06 GHz, 16 GB memory and 500 GB disk storage.
Each site holds one fragment of the dataset. At each site, we install gStore to find inner matches, since it supports the graph-based SPARQL evaluation paradigm.
We use MPICH-3.0.4 library for communication.

7.2 Exp1: Evaluating each stage’s performance

In this experiment, we study the performance of our system at each stage (i.e., partial evaluation and assembly process) with regard to different queries in WatDiv 1B and LUBM 1000.
下面是具體的report, snowflake——several stars linked by a path.
complex——a combination of the above with complex structure.

7.2.1 Partial evaluation

Tables 2 and 3 show that if there are some selective triple patterns10 in the query, the partial evaluation is much faster than others.
More inner matches and local partial matches lead to higher running time in the partial evaluation stage.

7.2.2 Assembly

We find that distributed assembly can beat the centralized one when there are lots of local partial matches and crossing matches.
大量的LPM送至server，server的處理能力達到了瓶頸。
大部分都涉及了所有的fragments。

7.3 Exp 2: Evaluating optimizations in assembly

In this experiment, we use WatDiv 1B to evaluate two different optimization techniques in the assembly: partitioning-based join strategy (Sect. 5.1) and the divide-and-conquer approach in the distributed assembly (Sect. 5.3).

7.3.1 Partitioning-based join

First, we compare partitioning-based join (i.e., Algorithm 3) with naive join processing (i.e., Algorithm 2) in Table 4, which shows that the partitioning-based strategy can greatly reduce the join cost.
Second, we evaluate the effectiveness of our cost model. 即最佳劃分的組裝比隨機劃分的組裝要快。

7.3.2 Divide-and-conquer in distributed assembly

Table 5 shows that dividing the search space will speed up distributed assembly.

7.4 Exp 3: Scalability test

In this experiment, we vary the RDF dataset size from 100 million triples (WatDiv 100M) to 1 billion triples (WatDiv 1B) to study the scalability of our methods.
Query response time is affected by both the increase in data size (which is 1x→10x in these experiments) and the query type.

7.5 Exp 4: Intermediate result size and query performance versus query decomposition approaches

Table 6 compares the number of intermediate results in our method with two typical query decomposition approaches, i.e., GraphPartition and TripleGroup.
More intermediate results typically lead to more assembly time. Table 7 shows that our query response time is faster than others.

Our technique is always faster regardless of the use of MPI or MapReduce-based join.
Our partial evaluation process is more expensive in evaluating local queries than GraphPartition and TripleGroup in many cases.
Our system generally outperforms GraphPartition and TripleGroup significantly if they use MapReduce-based join. Even when GraphPartition and TripleGroup use distributed joins, our system is still faster than them in most cases.

7.6 Exp 5: Performance on RDF datasets with one billion triples

This experiment is a comparative evaluation of our method against GraphPartition, TripleGroup and EAGRE on three very large RDF datasets with more than one billion triples, WatDiv 1B, LUBM 10000 and BTC.
一半左右的query沒有中間結果生成，所以差別不大。但其他的query本方法優勢明顯。
EAGRE stores all triples as flat files in HDFS and answers SPARQL queries by scanning the files. 所以比較耗時。但我們使用圖匹配去響應查詢，避免了對整個數據集的掃描。

7.7 Exp 6: Impact of different partitioning strategies

In this experiment, we test the performance under three different partitioning strategies over WatDiv 100 M.

We implement three partitioning strategies: uniformly distributed hash partitioning, exponentially distributed hash partitioning, and minimum-cut graph partitioning.
The first partitioning strategy uniformly hashes a vertex v in RDF graph G to a fragment.
The second strategy uses an exponentially distributed hash function with a rate parameter of 0.5.
Minimum-cut partitioning strategy generally leads to fewer crossing edges than the other two.
Although our partial evaluation and assembly framework is agnostic to the particular partitioning strategy, it is clear that it works better when fragment sizes are balanced, and the crossing edges are minimized.

7.8 Exp 7: Comparing with memory-based distributed RDF systems

We compare our approach (which is disk-based) against TriAD and Trinity.RDF that are memory-based distributed systems. Our system is faster.

7.9 Exp 8: Comparing with federated SPARQL systems

We compare our methods with some federated SPARQL query systems including (FedX and SPLENDID).

7.10 Exp 9: Comparing with centralized RDF systems

和RDF-3X在LUBM 10000數據集上作比較。

當查詢比較複雜的時候，我們的方法優於RDF-3X。如果查詢包含selective triple pattern，搜索空間比較小，則RDF-3X更快。

8 Conclusion

In this paper, we propose a graph-based approach to distributed SPARQL query processing that adopts the partial evaluation and assembly approach.
第一步，在每個段上對查詢Q進行評估找到local partial matches.
第二步，組裝這些local partial matches.兩種方法
集中式組裝：所有的local partial matches都發送至一個站點。
分佈式組裝：local partial matches在許多站點同時組裝。
本方法的優點有兩個：
一、本方法與分區方式無關，更加靈活。
二、中間結果比較少，和基於分區的方法相比。

下一步的工作：處理在linked open data (LOD)上的SPARQL queries。以及在分佈式RDF圖中的多重SPARQL查詢優化。

《Processing SPARQL queries over distributed RDF graphs》——讀書筆記