[Elasticsearch] 數據建模 - 處理關聯關係(1)

數據建模(Modeling Your Data)

ES是一頭不同尋常的野獸,尤其是當你來自SQL的世界時。它擁有很多優勢:性能,可擴展性,準實時的搜索,以及對大數據的分析能力。並且,它很容易上手!只需要下載就能夠開始使用它了。

但是它也不是魔法。爲了更好的利用ES,你需要了解它從而讓它能夠滿足你的需求。

在ES中,處理實體之間的關係並不像關係型存儲那樣明顯。在關係數據庫中的黃金準則 - 數據規範化,在ES中並不適用。在處理關聯關係嵌套對象父子關聯關係中,我們會討論幾種可行方案的優點和缺點。

緊接着在爲可擴展性而設計中,我們會討論ES提供的一些用來快速靈活實現擴展的特性。對於擴展,並沒有一個可以適用於所有場景的解決方案。你需要考慮數據是如何在你的系統中流轉的,從而恰當地對你的數據進行建模。針對基於時間的數據比如日誌事件或者社交數據流的方案比相對靜態的文檔集合的方案是十分不同的。

最後,我們會討論一樣在ES中不會擴展的東西。


處理關聯關係(Handling Relationships)

在真實的世界中,關聯關係很重要:博客文章有評論,銀行賬戶有交易,客戶有銀行賬戶,訂單有行項目,目錄也擁有文件和子目錄。

在關係數據庫中,處理關聯關係的方式讓你不會感到意外:

  • 每個實體(或者行,在關係世界中)可以通過一個主鍵唯一標識。
  • 實體是規範化了的。對於一個唯一的實體,它的數據僅被存儲一次,而與之關聯的實體則僅僅保存它的主鍵。改變一個實體的數據只能發生在一個地方。
  • 在查詢期間,實體可以被聯接(Join),它讓跨實體查詢成爲可能。
  • 對於單個實體的修改是原子性,一致性,隔離性和持久性的。(參考ACID事務獲取更多相關信息。)
  • 絕大多數的關係型數據庫都支持針對多個實體的ACID事務。

但是關係型數據庫也有它們的侷限,除了在全文搜索領域它們拙劣的表現外。在查詢期間聯接實體是昂貴的 - 聯接的實體越多,那麼查詢的代價就越大。對不同硬件上的實體執行聯接操作的代價太大以至於它甚至是不切實際的。這就爲在單個服務器上能夠存儲的數據量設下了一個限制。

ES,像多數NoSQL數據庫那樣,將世界看作是平的。一個索引就是一系列獨立文檔的扁平集合。一個單一的文檔應該包括用來判斷它是否符合一個搜索請求的所有信息。

雖然在ES中改變一份文檔的數據是符合ACIDic的,涉及到多份文檔的事務就不然了。在ES中,當事務失敗後是沒有辦法將索引回滾到它之前的狀態的。

這個扁平化的世界有它的優勢:

  • 索引是迅速且不需要上鎖的。
  • 搜索是迅速且不需要上鎖的。
  • 大規模的數據可以被分佈到多個節點上,因爲每份文檔之間是獨立的。

但是關聯關係很重要。我們需要以某種方式將扁平化的世界和真實的世界連接起來。在ES中,有4中常用的技術來管理關聯數據:

通常最終的解決方案會結合這些方案的幾種。


應用端聯接(Application-side Joins)

我們可以通過在應用中實現聯接來(部分)模擬一個關係型數據庫。比如,當我們想要索引用戶和他們的博客文章時。在關係型的世界中,我們可以這樣做:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /my_index/user/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>  (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">email</span>":    <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"[email protected]"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">dob</span>":      <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1970/10/24"</span>
</span>}

PUT /my_index/blogpost/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>":    <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">body</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"It's complicated..."</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>)
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>

(1)(2) 索引,類型以及每份文檔的ID一起構成了主鍵。

(3) 博文通過保存了用戶的ID來聯接到用戶。由於索引和類型是被硬編碼到了應用中的,所以這裏並不需要。

通過用戶ID等於1來找到對應的博文很容易:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">GET /my_index/blogpost/_search
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">filtered</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">filter</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> </span>}
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

爲了找到用戶John的博文,我們可以執行兩條查詢:第一條查詢用來得到所有名爲John的用戶的IDs,第二條查詢通過這些IDs來得到對應文章:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">GET /my_index/user/_search
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John"</span>
    </span>}
  </span>}
</span>}

GET /my_index/blogpost/_search
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">filtered</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">filter</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">terms</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>] </span>}   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul>

(1) 傳入到terms過濾器的值是第一條查詢的結果。

應用端聯接最大的優勢在於數據是規範化了的。改變用戶的名字只需要在一個地方操作:用戶對應的文檔。劣勢在於你需要在搜索期間運行額外的查詢來聯接文檔。

在這個例子中,只有一位用戶匹配了第一條查詢,但是在實際應用中可能輕易就得到了數以百萬計的名爲John的用戶。將所有的IDs傳入到第二個查詢中會讓該查詢非常巨大,它需要執行百萬計的term查詢。

這種方法在第一個實體的文檔數量較小並且它們很少改變時合適(這個例子中實體指的是用戶)。這就使得通過緩存結果來避免頻繁查詢成爲可能。


反規範化你的數據(Denormalizing Your Data)

讓ES達到最好的搜索性能的方法是採用更直接的辦法,通過在索引期間反規範化你的數據。通過在每份文檔中包含冗餘數據來避免聯接。

如果我們需要通過作者的名字來搜索博文,可以在博文對應的文檔中直接包含該作者的名字:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /my_index/user/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">email</span>":    <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"[email protected]"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">dob</span>":      <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1970/10/24"</span>
</span>}

PUT /my_index/blogpost/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>":    <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">body</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"It's complicated..."</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>":     <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">id</span>":       <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
    "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span> 
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li></ul>

現在,我們可以通過一條查詢來得到用戶名爲John的博文了:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">GET /my_index/blogpost/_search
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">bool</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">must</span>": <span class="hljs-value" style="box-sizing: border-box;">[
        { "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"relationships"</span> </span>}</span>},
        { "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">user.name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John"</span>          </span>}</span>}
      ]
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>

對數據的反規範化的優勢在於速度。因爲每份文檔包含了用於判斷是否匹配查詢的所有數據,不需要執行代價高昂的聯接操作。

字段摺疊(Field Collapsing)

一個常見的需求是通過對某個特定的字段分組來展現搜索結果。我們或許希望通過對用戶名分組來返回最相關的博文。對用戶名分組意味着我們需要使用到terms聚合。爲了對用戶的全名進行分組,name字段需要有not_analyzed的原始值,如聚合和分析中解釋的那樣。

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /my_index/_mapping/blogpost
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">properties</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">properties</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (1)
          "<span class="hljs-attribute" style="box-sizing: border-box;">type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
          "<span class="hljs-attribute" style="box-sizing: border-box;">fields</span>": <span class="hljs-value" style="box-sizing: border-box;">{
            "<span class="hljs-attribute" style="box-sizing: border-box;">raw</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (2)
              "<span class="hljs-attribute" style="box-sizing: border-box;">type</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
              "<span class="hljs-attribute" style="box-sizing: border-box;">index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"not_analyzed"</span>
            </span>}
          </span>}
        </span>}
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul>

(1) 字段用來支持全文搜索。 
(2) 字段用來支持terms聚合來完成分組。

然後添加一些數據:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /my_index/user/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">email</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"[email protected]"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">dob</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1970/10/24"</span>
</span>}

PUT /my_index/blogpost/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">body</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"It's complicated..."</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
    "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span>
  </span>}
</span>}

PUT /my_index/user/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Alice John"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">email</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"[email protected]"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">dob</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1979/01/04"</span>
</span>}

PUT /my_index/blogpost/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships are cool"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">body</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"It's not complicated at all..."</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span></span>,
    "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Alice John"</span>
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li></ul>

現在我們可以運行一個查詢來獲取關於relationships的博文,通過用戶名爲John對結果進行分組。這都要感謝top_hits聚合:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">GET /my_index/blogpost/_search?search_type=count (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (2)
    "<span class="hljs-attribute" style="box-sizing: border-box;">bool</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">must</span>": <span class="hljs-value" style="box-sizing: border-box;">[
        { "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"relationships"</span> </span>}</span>},
        { "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">user.name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John"</span>          </span>}</span>}
      ]
    </span>}
  </span>}</span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">aggs</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">users</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">terms</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">field</span>":   <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"user.name.raw"</span></span>,      (3) 
        "<span class="hljs-attribute" style="box-sizing: border-box;">order</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">top_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"desc"</span> </span>} (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>)
      </span>}</span>,
      "<span class="hljs-attribute" style="box-sizing: border-box;">aggs</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">top_score</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">max</span>":      <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">script</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_score"</span>           </span>}</span>}</span>,  (5)
        "<span class="hljs-attribute" style="box-sizing: border-box;">blogposts</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">top_hits</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">_source</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span></span>, "<span class="hljs-attribute" style="box-sizing: border-box;">size</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span> </span>}</span>}   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>)
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li></ul>

(1) 我們感興趣的博文在blogposts聚合中被返回了,因此我們可以通過設置search_type=count來禁用通常的搜索結果。

(2) 該查詢返回用戶名爲John,title匹配relationships的博文。

(3) terms聚合爲每個user.name.raw值創建一個桶。

(4)(5) 在users聚合中,使用top_score聚合通過每個桶中擁有最高分值的文檔進行排序。

(6) top_hits聚合只返回每個用戶的5篇最相關博文的title字段。

部分響應如下所示:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">...
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hits"</span>: {
  "<span class="hljs-attribute" style="box-sizing: border-box;">total</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">max_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">hits</span>":      <span class="hljs-value" style="box-sizing: border-box;">[]    (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
</span>},
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"aggregations"</span>: {
  "<span class="hljs-attribute" style="box-sizing: border-box;">users</span>": <span class="hljs-value" style="box-sizing: border-box;">{
     "<span class="hljs-attribute" style="box-sizing: border-box;">buckets</span>": <span class="hljs-value" style="box-sizing: border-box;">[
        {
           "<span class="hljs-attribute" style="box-sizing: border-box;">key</span>":       <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span></span>,    (2)
           "<span class="hljs-attribute" style="box-sizing: border-box;">doc_count</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
           "<span class="hljs-attribute" style="box-sizing: border-box;">blogposts</span>": <span class="hljs-value" style="box-sizing: border-box;">{
              "<span class="hljs-attribute" style="box-sizing: border-box;">hits</span>": <span class="hljs-value" style="box-sizing: border-box;">{    (3)
                 "<span class="hljs-attribute" style="box-sizing: border-box;">total</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
                 "<span class="hljs-attribute" style="box-sizing: border-box;">max_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.35258877</span></span>,
                 "<span class="hljs-attribute" style="box-sizing: border-box;">hits</span>": <span class="hljs-value" style="box-sizing: border-box;">[
                    {
                       "<span class="hljs-attribute" style="box-sizing: border-box;">_index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"my_index"</span></span>,
                       "<span class="hljs-attribute" style="box-sizing: border-box;">_type</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blogpost"</span></span>,
                       "<span class="hljs-attribute" style="box-sizing: border-box;">_id</span>":    <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"2"</span></span>,
                       "<span class="hljs-attribute" style="box-sizing: border-box;">_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.35258877</span></span>,
                       "<span class="hljs-attribute" style="box-sizing: border-box;">_source</span>": <span class="hljs-value" style="box-sizing: border-box;">{
                          "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships"</span>
                       </span>}
                    </span>}
                 ]
              </span>}
           </span>}</span>,
           "<span class="hljs-attribute" style="box-sizing: border-box;">top_score</span>": <span class="hljs-value" style="box-sizing: border-box;">{    (4)
              "<span class="hljs-attribute" style="box-sizing: border-box;">value</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.3525887727737427</span>
           </span>}
        </span>},
...</span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li></ul>

(1) hits數組爲空因爲我們設置了search_type=count

(2) 針對每個用戶都有一個對應的桶。

(3) 在每個用戶的桶中,有一個blogposts.hits數組,它包含了該用戶的相關度最高的搜索結果。

(4) 用戶桶通過用戶最相關的博文進行排序。

使用top_hits聚合等效於運行獲取最相關博文及其對應用戶的查詢,然後針對每個用戶運行同樣的查詢,來得到每個用戶最相關的博文。可見使用top_hits更高效。

每個桶中返回的top hits是通過運行一個基於原始主查詢的迷你查詢而來。該迷你查詢也同樣支持高亮(Highlighting)以及分頁(Pagination)。


反規範化和併發(Denormalization and Concurrency)

當然,數據反規範化也有弊端。首先,它會讓索引變大,因爲每篇博文的_source都變大了,與此同時需要索引的字段也變多了。通常這並不是一個大問題。寫入到磁盤的數據會被高度壓縮,而且磁盤存儲空間也不貴。ES能夠很從容地處理這些多出來的數據。

更重要的弊端在於,如果用戶修改了他的名字,他名下的所有博文都需要被更新。即使用戶真的這麼做了,一位用戶也不太可能寫了上千篇博文,因此通過scrollbulk APIs,更新也不會超過1秒。

然而,讓我們來考慮一個變化更常見,影響更深遠和重要的複雜情景 - 併發。

在這個例子中,我們通過ES來模擬一個擁有目錄樹的文件系統,就像Linux上的文件系統那樣:根目錄是/,每個目錄都能包含文件和子目錄。

我們希望能夠搜索某個目錄下的文件,就像下面這樣:

grep "some text" /clinton/projects/elasticsearch/*

它需要我們對文件的路徑進行索引:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/file/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"README.txt"</span></span>, 
  "<span class="hljs-attribute" style="box-sizing: border-box;">path</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton/projects/elasticsearch"</span></span>, 
  "<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Starting a new Elasticsearch project is easy..."</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

NOTE

說實在的,我們同時也應該一個目錄下所有的文件和子目錄進行索引,但是爲了簡潔起見,這裏忽略了這一需求。

我們還需要能夠搜索某個目錄下任意深度的文件,就像下面這樣:

grep -r "some text" /clinton

爲了支持這一需求,需要對路徑層次進行索引:

  • /clinton
  • /clinton/projects
  • /clinton/projects/elasticsearch

該層次結構可以通過對path字段使用path_hierarchy tokenizer來自動生成:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">settings</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">analysis</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">analyzer</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">paths</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (1)
          "<span class="hljs-attribute" style="box-sizing: border-box;">tokenizer</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"path_hierarchy"</span>
        </span>}
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li></ul>

(1) 以上的自定義的分析器使用了path_hierarchy分詞器,使用其默認設置。參見path_hierarchy tokenizer

文件類型的映射則像下面這樣:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/_mapping/file
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">properties</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (1)
      "<span class="hljs-attribute" style="box-sizing: border-box;">type</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
      "<span class="hljs-attribute" style="box-sizing: border-box;">index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"not_analyzed"</span>
    </span>}</span>,
    "<span class="hljs-attribute" style="box-sizing: border-box;">path</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (2)
      "<span class="hljs-attribute" style="box-sizing: border-box;">type</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
      "<span class="hljs-attribute" style="box-sizing: border-box;">index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"not_analyzed"</span></span>,
      "<span class="hljs-attribute" style="box-sizing: border-box;">fields</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">tree</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (3)
          "<span class="hljs-attribute" style="box-sizing: border-box;">type</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
          "<span class="hljs-attribute" style="box-sizing: border-box;">analyzer</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"paths"</span>
        </span>}
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul>

(1) name字段會包含完整的名字。

(2)(3) path字段會包含完整的目錄名,而path.tree字段會包含路徑層次結構。

一旦建立了該索引並完成了文件的索引,我們就能夠執行如下查詢,它搜索/client/projects/elasticsearch目錄下包含有elasticsearch的文件:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">GET /fs/file/_search
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">filtered</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{
          "<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"elasticsearch"</span>
        </span>}
      </span>}</span>,
      "<span class="hljs-attribute" style="box-sizing: border-box;">filter</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (1)
          "<span class="hljs-attribute" style="box-sizing: border-box;">path</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton/projects/elasticsearch"</span>
        </span>}
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

(1) 只在該目錄下搜索。

任何存儲於/clinton目錄下的文件都會在path.tree字段中包含/clinton。因此我們可以通過下面的搜索來得到/clinton目錄下的所有文件:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">GET /fs/file/_search
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">filtered</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{
          "<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"elasticsearch"</span>
        </span>}
      </span>}</span>,
      "<span class="hljs-attribute" style="box-sizing: border-box;">filter</span>": <span class="hljs-value" style="box-sizing: border-box;">{
        "<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (1)
          "<span class="hljs-attribute" style="box-sizing: border-box;">path.tree</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton"</span>
        </span>}
      </span>}
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

(1) 尋找該目錄或其下任意子目錄中的文件。

重命名文件和目錄(Renaming Files and Directories)

到目前爲止還不錯。文件重命挺簡單 - 只需要一個簡單的更新或者索引請求就行了。你甚至可以使用樂觀併發控制(Optimistic Concurrency Control)來確保你的更改不會和另一個用戶的更改發生衝突。

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/file/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>?version=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>    (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">name</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"README.asciidoc"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">path</span>":     <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton/projects/elasticsearch"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Starting a new Elasticsearch project is easy..."</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

(1) version值能夠確保只有當索引中的文檔擁有相同的version值時,更改纔會生效。

我們還可以對目錄重命名,但是這意味着對該目錄下的所有文件執行更新。這個操作或快或慢,取決於有多少文件需要被更新。我們需要做的是使用scan-and-scroll來獲取所有的文件,然後使用bulk API完成更新。這個過程並不是原子性的,但是所有的文件都能夠被迅速地更新。


解決併發問題(Solving Concurrency Issues)

當我們允許多個用戶同時對文件和目錄進行重命名時,問題就來了。假設你對/clinton目錄進行重命名,它包含了成百上千個文件。同時,另外一個用戶重命名了/clinton/projects/elasticsearch/README.txt這個文件。該用戶的更改雖然在你的操作之後,但是它完成的也許更快。

下面兩種情況中的一種會發生:

  • 你決定使用version值,這意味着你對文件README.txt的重命名操作會因爲version衝突而失敗。
  • 你不使用versioning,那麼你的更改會直接覆蓋掉另一用戶的更改。

問題的根源在ES並不支持ACID事務。對單個文檔的更新雖然是符合ACID原則的,但是對多個文檔的更新則不然。

如果你使用的主要數據源是關係行數據庫,而ES只是簡單地被用來當作搜索引擎或者提升性能的方法,那麼首先更新數據庫,然後待這些更新操作成功後再將這些變更復制到ES中。這樣的話,你就能受益於數據庫對於ACID事務的支持了,它能保證ES中的所有更新順序都是正確的。併發的問題在關係型數據庫中被處理了。

如果你沒有使用關係型數據源,那麼這些併發問題就需要在ES中完成。下面有三種可行的方案,這些方案都涉及到了某種形式的鎖:

  • 全局鎖(Global Locking)
  • 文檔鎖(Document Locking)
  • 樹鎖(Tree Locking)

TIP

以上提到的方案都可以通過應用了相同原則的外部系統實現。

全局鎖(Global Locking)

我們可以通過在任何時候只允許一個進程執行更新操作來完全避免併發性的問題。大多數的修改只設計很少的幾個文件,完成的也相當快。對於頂層目錄的重命名也許會阻塞其它變更操作更久一點,但是這種情況發生的頻率也會更低一些。

因爲在ES中文檔級別的變更是滿足ACID的,我們可以將一份文檔是否存在作爲一個全局鎖。爲了獲取一個鎖,我們嘗試去創建一個全局鎖文檔:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/global/_create
{}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

如果上述的創建請求由於發生了衝突而失敗了,就表明另一個進程已經被授權得到了全局鎖,我們只能稍後重試。如果上述請求成功了,我們就擁有了全局鎖從而可以進行後續的變更操作。一旦這些操作完成了,必須通過刪除全局鎖文檔來釋放它:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">DELETE /fs/lock/global</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

取決於變更的頻繁程度和它們需要消耗的時間,全局鎖對系統的性能或許會有相當程度的限制。可以通過實現更加細粒度的鎖來增加並行性。

文檔鎖(Document Locking)

相比於對整個文件系統上鎖,我們可以通過上面提到的技術來對單個文檔完成鎖定。一個進程可以使用scan-and-scroll請求來獲取到變更會影響到的所有文檔的IDs,然後對每份文檔創建一個鎖文件:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/_bulk
{ "<span class="hljs-attribute" style="box-sizing: border-box;">create</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>}</span>}   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{ "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>    </span>}   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
{ "<span class="hljs-attribute" style="box-sizing: border-box;">create</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span></span>}</span>}
{ "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>    </span>}
...</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

(1) 鎖文檔的ID需要和被鎖定的文檔的ID一致。

(2) process_id是將要執行變更操作進程的唯一ID。

如果某些文檔已經被鎖了,那麼部分bulk請求會失敗,只好重試。

當然,如果我們試圖再次去鎖定所有的文檔,對於那些已經被我們鎖定的文檔,前面使用的創建語句會失敗!相比一個簡單的創建語句,我們需要使用帶有upsert參數的update請求:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">if ( ctx._source.process_id != process_id ) {   (1)
  assert false;   (2)
}
ctx.op = 'noop';  (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(1) process_id是我們傳入到腳本中的參數。

(2) assert false會拋出一個異常,它導致更新失敗。

(3) 將op從update修改爲noop能夠防止update請求真的執行變更操作,而仍然返回成功。

完整的update請求如下所示:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/lock/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/_update
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">upsert</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span> </span>}</span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"if ( ctx._source.process_id != process_id )
  { assert false }; ctx.op = 'noop';"</span>
  <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"params"</span>: {
    "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

如果文檔並不存在,upsert會執行insert操作 - 就和之前使用的create請求一樣。但是,如果文檔存在,腳本會查看文檔中保存的process_id。如果它和我們的相同,就會放棄update(noop)並返回成功。如果它和我們的不同,那麼assert false就會拋出一個異常告訴我們鎖定失敗了。

一旦所有的鎖都別成功創建了,重命名操作就開始了。在這之後,我們必須釋放所有的鎖,通過delete-by-query請求來完成:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/_refresh   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)

DELETE /fs/lock/_query
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
    "<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{
      "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>
    </span>}
  </span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

(1) refresh調用保證了所有的鎖文檔對delete-by-query請求是可見的。

文檔級別的鎖擁有更加細粒度的訪問控制,但是爲百萬計的文檔創建鎖是非常昂貴的。在某些場合下,比如前面例子中出現的目錄樹,可以通過更少的工作來達到細粒度的鎖定。

樹鎖(ree Locking)

相比像前面那樣對每份文檔上鎖,也可以支隊目錄樹的部分上鎖。我們需要對重命名的文檔或者目錄擁有獨佔性訪問,可以通過獨佔性鎖文檔實現:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"exclusive"</span> </span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

同時我們也需要對上層目錄使用分享鎖:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">{
  "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"shared"</span></span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">lock_count</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(1) lock_count記錄了擁有該分享鎖的進程數量。

對/clinton/projects/elasticsearch/README.txt重命名的進程需要該文件的獨佔鎖,以及針對目錄/clinton,/clinton/projects和/clinton/projects/elasticsearch的分享鎖。

對於獨佔鎖,可以通過簡單的創建請求來實現,但是分享鎖需要帶有腳本的update請求來實現:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">if (ctx._source.lock_type == 'exclusive') {
  assert false;   (1)
}
ctx._source.lock_count++   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(1) 如果lock_type是獨佔性的,那麼assert語句會拋出一個異常,導致update請求失敗。

(2) 否則,增加lock_count。

該腳本能夠處理當鎖文檔已經存在的情況,但是我們仍然需要使用upsert來處理它不存在時的情況。完整的update請求如下所示:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton/_update   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">upsert</span>": <span class="hljs-value" style="box-sizing: border-box;">{   (2)
    "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>":  <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"shared"</span></span>,
    "<span class="hljs-attribute" style="box-sizing: border-box;">lock_count</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
  </span>}</span>,
  "<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"if (ctx._source.lock_type == 'exclusive')
  { assert false }; ctx._source.lock_count++"</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

(1) 文檔的ID是/clinton,再進行URL編碼後變成了%2fclinton。

(2) 當文檔不存在時,upsert代表的文檔會被插入。

一旦我們成功獲取了文件所在目錄的所有上級目錄的分享鎖後,就可以嘗試去創建一個針對該文件的獨佔鎖了:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fprojects%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>felasticsearch%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fREADME.txt/_create
{ "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"exclusive"</span> </span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

現在,如果另外的某個人想要對/clinton目錄重命名,他們就需要獲取該目錄的獨佔鎖:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton/_create
{ "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"exclusive"</span> </span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

該請求會失敗,因爲一個擁有相同ID的鎖文檔已經存在了。該用戶只好等待我們的操作完成並釋放鎖。獨佔鎖可以被刪除:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">DELETE /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fprojects%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>felasticsearch%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fREADME.txt</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

而分享鎖需要另一個腳本來減少lock_count的計數,如果該計數減少到了0,就刪除它:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">if (--ctx._source.lock_count == <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>) {
  ctx.op = 'delete'   (1)
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

(1) 一旦lock_count爲0,ctx.op就從update變成delete。

對每個上級目錄都需要以相反的順序執行update請求,從最長的目錄到最短的目錄:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fprojects%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>felasticsearch/_update
{
  "<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"if (--ctx._source.lock_count == 0) { ctx.op = 'delete' } "</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

樹鎖通過最少的代價實現了細粒度的併發控制。當然,它並不能適用於每個場景 - 數據模型必須要有類似目錄樹這種結構纔行。

NOTE

以上的三種方案 - 全局鎖,文檔鎖和樹鎖 - 都沒有處理關於鎖的最棘手的問題:如果持有鎖的進程掛了怎麼辦?

一個進程的意外掛掉給我們留下了兩個問題:

  • 我們如何知道我們可以釋放被掛掉的進程持有的鎖?
  • 我們如何清理掛掉的進程沒有完成的工作?

這些話題都超出了本書的範疇,但是如果你決定使用鎖,就需要考慮一下它們。

儘管反規範化對於很多項目而言都是一個好的選擇,由於需要鎖的支持,會導致較爲複雜的實現。相比之下,ES提供了另外兩種模型來處理關聯的實體:嵌套對象(Nested Objects)和父子關係(Parent-Child Relationship)


發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章